Lessons learned in a decade of research software engineering GPU applications