Efficient and portable multi-tasking for heterogeneous systems