As mentioned in #452, here at Veo Technogies we are currently trying to integrate Kompute with a GStreamer video processing pipeline.
GStreamers Vulkan library provides an abstraction over Vulkan Images and Buffers that also tracks some state needed to accurately insert pipeline barriers and such, similarly to what Kompute does. With the kp::Memory base class it is easy for us to provide a memory subclass that wraps, for example, a GstVulkanBufferMemory structure, and then implement the various virtual methods to correctly to both work for Kompute and update the state in the the GStreamer structure.
However, in the video processing pipeline each frame would now be a new kp::Memory instance and since kp::Algorithm takes ownership of the memory objects we would - as it currently is - need to create a new instance of the algorithm for each frame.
As I understand the Kompute abstractions, I think that the algorithm is an abstraction over the compute shader, the compute pipeline, the descriptor layout and the descriptor set. One way forward could be to add an “update memory objects” method to kp::Algorithm that would either update the descriptor set, or create a new one from a pool.
For our use-case a using multiple descriptor sets would be preferable, since it would allow us to run multiple instances of the same algorithm asynchronously, instead of having to wait for processing of one frame to end before starting the next. This would require some sort of cleanup mechanism, though with a list of fences to check occasionally to know when descriptor sets can be returned to the pool (I guess this is same issue as #214).
As mentioned in #452, here at Veo Technogies we are currently trying to integrate Kompute with a GStreamer video processing pipeline.
GStreamers Vulkan library provides an abstraction over Vulkan Images and Buffers that also tracks some state needed to accurately insert pipeline barriers and such, similarly to what Kompute does. With the
kp::Memorybase class it is easy for us to provide a memory subclass that wraps, for example, aGstVulkanBufferMemorystructure, and then implement the various virtual methods to correctly to both work for Kompute and update the state in the the GStreamer structure.However, in the video processing pipeline each frame would now be a new
kp::Memoryinstance and sincekp::Algorithmtakes ownership of the memory objects we would - as it currently is - need to create a new instance of the algorithm for each frame.As I understand the Kompute abstractions, I think that the algorithm is an abstraction over the compute shader, the compute pipeline, the descriptor layout and the descriptor set. One way forward could be to add an “update memory objects” method to
kp::Algorithmthat would either update the descriptor set, or create a new one from a pool.For our use-case a using multiple descriptor sets would be preferable, since it would allow us to run multiple instances of the same algorithm asynchronously, instead of having to wait for processing of one frame to end before starting the next. This would require some sort of cleanup mechanism, though with a list of fences to check occasionally to know when descriptor sets can be returned to the pool (I guess this is same issue as #214).