update docker - #1225
Conversation
#934) rs2v shot infer: 1. stream save video 2. support deploy worker 3. va_controller fix div 0 error
Co-authored-by: wangshankun <wangshankun2011@hotmail.com>
This pull request refactors the `DataManager` class and related threading logic in `lightx2v/disagg/conn.py` to better support multi-room (multi-session) operation, improve thread management, and enhance code clarity. The changes introduce per-room resource management, refactor thread lifecycle handling, and add a new `ReqManager` utility class for ZeroMQ-based message passing. Key changes include: **Multi-room support and resource management:** * Refactored `DataManager` to manage per-room resources such as `data_args`, threads, and events, enabling concurrent handling of multiple rooms/sessions. Initialization and cleanup for each room are now handled through new `init` and `release` methods. * Updated data transfer and synchronization methods to use per-room arguments and events, ensuring correct operation in multi-room scenarios. [[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R200-R217) [[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L149-R253) [[3]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L331-R466) [[4]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L379-R519) [[5]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L392-R535) **Thread lifecycle and event handling improvements:** * Replaced global thread/event management with per-room thread pools and stop events. Added helper methods to start, register, and end threads for each room, allowing for safe cleanup and improved robustness. [[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R78-R191) [[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L187-R284) [[3]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R300) [[4]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L222-R361) [[5]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L268-R392) [[6]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R408) [[7]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L303-R449) **API and naming consistency:** * Updated method signatures and variable names for clarity and consistency, such as using `sender_data_ptrs` and `receiver_ptrs` instead of ambiguous names. [[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R200-R217) [[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L149-R253) * Improved error handling and input validation, for example by requiring `bootstrap_room` in `DataReceiver`. **New utility class:** * Added `ReqManager`, a utility class for sending and receiving Python objects over ZeroMQ, including support for non-blocking receives and automatic conversion of nested mappings to built-in types. **Minor improvements:** * Three services utilize queues to maintain request states. * Dependencies between requests exist solely during two message communication phases, while message transmission is handled asynchronously. * All operations within the orange blocks support batch processing. These changes collectively improve the scalability, maintainability, and clarity of the data disaggregation infrastructure.
support wan 2.1 in intel ptl feature: offloading and feature caching. --------- Co-authored-by: helloyongyang <yongyang1030@163.com>
Co-authored-by: gushiqiao <975033167>
This pull request significantly refactors the data transfer and threading logic in the `DataManager` class in `lightx2v/disagg/conn.py` to improve thread safety, resource management, and scalability. The changes introduce a global transfer thread (instead of per-room transfer threads), per-room ZeroMQ sockets, and improved locking for shared state. The code is now better structured for concurrent operations and easier resource cleanup. Key changes include: **Threading and Synchronization Improvements** - Introduced a single global transfer thread managed by `DataManager`, replacing per-room transfer threads. This thread is controlled by new `transfer_event` and `transfer_stop_event` attributes, and is started only for relevant disaggregation phases/modes. (`[lightx2v/disagg/conn.pyR87-L127](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R87-L127)`) - Added a `pool_lock` to synchronize access to shared pools (`request_pool`, `request_status`, `waiting_pool`), ensuring thread safety during concurrent operations. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R87-L127)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R470)`, `[[3]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L464-R483)`, `[[4]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R496)`) **ZeroMQ Socket Management** - Replaced a single shared `server_socket` with per-room sockets managed in the new `room_sockets` dictionary. Added helper methods to create, bind, and close sockets for each room, enabling independent communication channels per room and preventing port conflicts. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R87-L127)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R253-L190)`) - Updated all socket usage in phase threads to use the correct per-room socket. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L238-R330)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L251-R340)`, `[[3]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L346-R400)`, `[[4]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L359-R410)`, `[[5]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R426-R451)`) **Resource Cleanup and API Changes** - Refactored the `release` method to clean up all threads, events, sockets, and data pointers across all rooms, ensuring a clean shutdown. The old per-room `release` is renamed to `remove`. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L136-R186)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R210-R240)`) - Added socket cleanup to `end_room_threads` to avoid resource leaks. (`[lightx2v/disagg/conn.pyR253-L190](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R253-L190)`) **Port Calculation and Addressing** - Updated port calculation for both sender and receiver sockets to include a `room * 10` offset, preventing port collisions when multiple rooms are used simultaneously. Also updated status synchronization to use the new port scheme. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L228-R318)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L238-R330)`, `[[3]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L346-R400)`, `[[4]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L517-R534)`) **Minor API and Logic Adjustments** - Simplified the `DataSender` and `DataReceiver` initialization and removed unused arguments. (`[lightx2v/disagg/conn.pyL496-R513](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7L496-R513)`) - Removed redundant or obsolete per-room transfer event logic in favor of the global transfer event. (`[[1]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R87-L127)`, `[[2]](diffhunk://#diff-a5e5778ac7adc9b8f2c175153e932db47158abebba325b47298001bc80e89ba7R253-L190)`) **New Features** - Use a round-robin policy to dispatch requests to instances. These changes collectively make the data transfer subsystem more robust, scalable, and maintainable.
Co-authored-by: gushiqiao <975033167>
Co-authored-by: helloyongyang <yongyang1030@163.com>
This pull request introduces several new configuration files and implements a monitoring and reporting system for the disaggregated LightX2V services. It also improves service initialization in example scripts to ensure proper configuration is passed to each service instance. **Key changes:** Configuration for disaggregated services: - Added four new JSON configuration files for different service roles: encoder, transformer, decoder, and controller. These files specify parameters such as inference steps, video dimensions, attention types, and disaggregation settings. [[1]](diffhunk://#diff-41d2f7220c63c963f3ff3e8377496b594d63faa693bb1e38ffc18dfd29938845R1-R26) [[2]](diffhunk://#diff-5f43adcf76ffff0c1bc538539fba7c370b4e0a943d2804c2e23b1156a653b784R1-R26) [[3]](diffhunk://#diff-4192d6b03bc4501c96c4803d7da7637573a7fac01e8a2484578fd5450935a217R1-R26) [[4]](diffhunk://#diff-05d0c33fedf1f64ba265e72d011bb1d4a2f2a85596edcdd5e88fe57432f56931R1-R26) Monitoring and reporting system: - Introduced a new `monitor.py` module that provides a `Reporter` class for reporting GPU metrics (utilization, memory usage) and a `Monitor` class for polling these metrics from multiple service nodes using ZeroMQ. - Added a constant `MONITOR_POLLING_PORT` to `conn.py` for standardized monitoring communication. Service initialization improvements: - Updated example scripts (`wan_t2v_service.py`, `wan_i2v_service.py`) to ensure that the `EncoderService`, `TransformerService`, and `DecoderService` are instantiated with the required configuration object. [[1]](diffhunk://#diff-6a1e9b2bd29fe1161419c7e0fe72430075447668bb92dd7365d41eaf3526deedL70-R84) [[2]](diffhunk://#diff-6bb32514e1f16ef26c1fb92c5c9af098e25c33f915c0d0b8eeabad865c081009L87-R101) - Added a new example script `run_service.py` that provides a unified entry point for running any of the disaggregated services based on command-line arguments and configuration files. This script normalizes configuration, resolves service roles, and initializes the appropriate service. Decentralized queue scheduling: - Last stage instances produce requests to an RDMA buffer, next stage instances consume requests from it. - Each instance uses RDMA one-sided verbs to operate the RDMA buffer.
Restores the full Stage 1 sigma schedule in init_run() before each job so long-lived workers do not keep the upsampler's shorter sigma schedule (which left Stage 1 at 3 steps instead of 8 after a prior Stage 2 run).
Co-authored-by: chendingyu <chendingyu1@sensetime.com>
Co-authored-by: chendingyu <chendingyu1@sensetime.com>
Co-authored-by: yihuiwen <yihuiwen@sensetime.com>
Co-authored-by: yihuiwen <yihuiwen@sensetime.com>
support 1080p sekotalk
#1202) Co-authored-by: yihuiwen <yihuiwen@sensetime.com>
Co-authored-by: yihuiwen <yihuiwen@sensetime.com>
## Summary Add end-to-end LightX2V inference support for Cosmos3 Super / Cosmos3 Super Omni tasks, with configs and scripts aligned by task name. ## Supported Tasks - `t2i` / `cosmos3_super_t2i` - Text-to-image generation from a text prompt. - `t2v` / `cosmos3_super_omni_t2v` - Text-to-video generation from a text prompt. - `i2v` / `cosmos3_super_i2v`, `cosmos3_super_omni_i2v` - Image-to-video generation conditioned on an input first frame plus prompt. - `t2av` / `cosmos3_super_omni_t2av` - Text-to-audio-video generation from a prompt, producing video with generated audio. - `i2av` / `cosmos3_super_omni_i2av` - Image-to-audio-video generation from a first frame and prompt, producing video with generated audio. - `i2va` forward dynamics / `cosmos3_super_omni_action_fd_agibotworld` - Action-conditioned video rollout from an initial observation image and a provided robot action chunk. - `i2va` multi-chunk forward dynamics / `cosmos3_super_omni_action_fd_agibotworld_multichunk` - Autoregressive multi-segment action rollout: each generated segment feeds its last frame into the next segment, using subsequent action chunks. - `v2av` inverse dynamics / `cosmos3_super_omni_action_id_av` - Inverse dynamics for the autonomous-driving domain: condition on an observed video and predict the corresponding action sequence, saving action output to JSON. ## Implementation Notes - Reuses the Cosmos3 runner/model path instead of importing diffusers model code. - Adds action conditioning support for forward dynamics, inverse dynamics, and multi-chunk rollout. - Adds audio decoding/muxing support for Omni audio-video tasks. - Aligns Cosmos3 config filenames with the script names under `scripts/cosmos3`. - Keeps per-task configs and scripts consistent with existing LightX2V style. ## Validation - Verified Cosmos3 config JSON files parse successfully. - Verified Cosmos3 shell scripts reference existing matching config files. - Verified updated Cosmos3 runner Python syntax compiles successfully.
There was a problem hiding this comment.
Code Review
This pull request introduces a new Conda-based Dockerfile (Dockerfile_cu130_conda) and updates existing Dockerfiles to adjust dependency installations. Key feedback focuses on optimizing the Docker image size by using --no-cache-dir for large packages, avoiding editable installs (-e) for custom attention kernels, and cleaning up cloned repositories after installation. Additionally, there are recommendations to remove the standalone bson package to prevent namespace clashes with pymongo, fix the scoping of the MAX_JOBS environment variable during the uv build step, and correct an inaccurate Python version comment in the ROS2 Jazzy Dockerfile.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
|
||
| RUN pip install --no-cache-dir packaging ninja cmake scikit-build-core uv meson ruff pre-commit fastapi uvicorn requests -U | ||
|
|
||
| RUN pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cu130 |
There was a problem hiding this comment.
PyTorch, torchvision, and torchaudio are extremely large packages. Installing them without --no-cache-dir causes pip to cache the downloaded wheels in /root/.cache/pip, which unnecessarily doubles the required disk space and significantly increases the final Docker image size. The same applies to vllm and sglang-kernel on lines 31 and 33.
RUN pip install --no-cache-dir torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cu130
|
|
||
| RUN pip install --no-cache-dir jsonschema pymongo mooncake-transfer-engine tos zmq graphviz pybind11 | ||
|
|
||
| RUN pip install --no-cache-dir bson aio-pika PyJWT asyncpg>=0.27.0 aioboto3>=12.0.0 alibabacloud_dypnsapi20170525==1.2.2 redis==6.4.0 |
There was a problem hiding this comment.
Installing the standalone bson package from PyPI alongside pymongo will cause a name clash and overwrite pymongo's internal bson module, leading to runtime import errors (e.g., AttributeError when using pymongo). Since pymongo already includes its own bson implementation, you should remove bson from this pip install command.
RUN pip install --no-cache-dir aio-pika PyJWT asyncpg>=0.27.0 aioboto3>=12.0.0 alibabacloud_dypnsapi20170525==1.2.2 redis==6.4.0
| RUN pip install --no-cache-dir https://github.com/windreamer/flash-attention3-wheels/releases/download/2026.05.11-5e0e3b1/flash_attn_3-3.0.0%2B20260511.cu130torch2110cxx11abitrue.ab6632-cp39-abi3-linux_x86_64.whl | ||
|
|
||
| # FA4 build from source: https://github.com/Dao-AILab/flash-attention | ||
| RUN git clone https://github.com/Dao-AILab/flash-attention.git --recursive && cd flash-attention && pip install -e "flash_attn/cute[dev,cu13]" |
There was a problem hiding this comment.
Installing flash-attention in editable mode (-e) prevents you from deleting the cloned repository, which leaves a large amount of unnecessary source files in the Docker image. Since this is a production/runtime image, you should install it normally and clean up the cloned repository to reduce the image size.
RUN git clone https://github.com/Dao-AILab/flash-attention.git --recursive && \n cd flash-attention && \n pip install --no-cache-dir "flash_attn/cute[dev,cu13]" && \n cd .. && \n rm -rf flash-attention
| RUN git clone https://github.com/ModelTC/SageAttention.git --depth 1 | ||
|
|
||
| RUN cd SageAttention && \ | ||
| CUDA_ARCHITECTURES="8.0,8.6,8.9,9.0,12.0" \ | ||
| EXT_PARALLEL=4 \ | ||
| NVCC_APPEND_FLAGS="--threads 8" \ | ||
| MAX_JOBS=32 \ | ||
| pip install --no-cache-dir --no-build-isolation -v -e . |
There was a problem hiding this comment.
Installing SageAttention in editable mode (-e) prevents cleaning up the cloned repository. Installing it normally and removing the cloned directory will significantly reduce the Docker image size.
RUN git clone https://github.com/ModelTC/SageAttention.git --depth 1 && \n cd SageAttention && \n CUDA_ARCHITECTURES="8.0,8.6,8.9,9.0,12.0" \n EXT_PARALLEL=4 \n NVCC_APPEND_FLAGS="--threads 8" \n MAX_JOBS=32 \n pip install --no-cache-dir --no-build-isolation -v . && \n cd .. && \n rm -rf SageAttention
| RUN git clone https://github.com/ModelTC/SageAttention-1104.git --depth 1 | ||
|
|
||
| RUN cd SageAttention-1104/sageattention3_blackwell && python setup.py install && rm -rf build |
There was a problem hiding this comment.
The cloned SageAttention-1104 repository is not cleaned up after installation, which unnecessarily increases the Docker image size. You should remove the directory at the end of the RUN instruction.
RUN git clone https://github.com/ModelTC/SageAttention-1104.git --depth 1 && \n cd SageAttention-1104/sageattention3_blackwell && \n python setup.py install && \n cd /app && \n rm -rf SageAttention-1104
| RUN git clone https://github.com/NVIDIA/cutlass.git --depth 1 && cd /app/lightx2v_kernel && MAX_JOBS=32 && CMAKE_BUILD_PARALLEL_LEVEL=4 \ | ||
| uv build --wheel \ | ||
| -Cbuild-dir=build . \ | ||
| -Ccmake.define.CUTLASS_PATH=/app/cutlass \ | ||
| --verbose \ | ||
| --color=always \ | ||
| --no-build-isolation \ | ||
| && pip install dist/*whl --force-reinstall --no-deps \ | ||
| && rm -rf /app/lightx2v_kernel && rm -rf /app/cutlass |
There was a problem hiding this comment.
The environment variable MAX_JOBS=32 is separated from the uv build command by &&, which means it is treated as a separate shell command and will not be exported to the environment of uv build. To ensure MAX_JOBS is correctly passed to the build process, it should directly prefix the command or be exported. Also, adding --no-cache-dir to the pip install command is recommended to avoid caching the built wheel.
RUN git clone https://github.com/NVIDIA/cutlass.git --depth 1 && cd /app/lightx2v_kernel && \n MAX_JOBS=32 CMAKE_BUILD_PARALLEL_LEVEL=4 uv build --wheel \n -Cbuild-dir=build . \n -Ccmake.define.CUTLASS_PATH=/app/cutlass \n --verbose \n --color=always \n --no-build-isolation \n && pip install --no-cache-dir dist/*whl --force-reinstall --no-deps \n && rm -rf /app/lightx2v_kernel /app/cutlass
| RUN git clone --depth 1 https://github.com/linux-rdma/rdma-core.git && \ | ||
| cd rdma-core/ && \ | ||
| mkdir build-ibv57 && \ | ||
| cd build-ibv57 && \ | ||
| cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DNO_MAN_PAGES=1 && \ | ||
| make -j"$(nproc)" && \ | ||
| make install && \ | ||
| ldconfig |
There was a problem hiding this comment.
The cloned rdma-core repository and its build directory are not cleaned up after installation, which unnecessarily increases the Docker image size. You should remove the rdma-core directory at the end of the RUN instruction.
RUN git clone --depth 1 https://github.com/linux-rdma/rdma-core.git && \n cd rdma-core/ && \n mkdir build-ibv57 && \n cd build-ibv57 && \n cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DNO_MAN_PAGES=1 && \n make -j"$(nproc)" && \n make install && \n ldconfig && \n cd /app && \n rm -rf rdma-core
| RUN git clone https://github.com/ModelTC/q8_kernels.git --depth 1 | ||
| RUN cd q8_kernels && git submodule init && git submodule update && python setup.py install && rm -rf build |
There was a problem hiding this comment.
The cloned q8_kernels repository is not cleaned up after installation, which unnecessarily increases the Docker image size. You should remove the directory at the end of the RUN instruction.
RUN git clone https://github.com/ModelTC/q8_kernels.git --depth 1 && \n cd q8_kernels && \n git submodule init && \n git submodule update && \n python setup.py install && \n cd .. && \n rm -rf q8_kernels
| RUN git clone https://github.com/ModelTC/SpargeAttn.git --depth 1 | ||
| RUN cd SpargeAttn && TORCH_CUDA_ARCH_LIST="8.0;8.6;8.9;9.0" pip install --no-cache-dir --no-build-isolation -v -e . |
There was a problem hiding this comment.
Installing SpargeAttn in editable mode (-e) prevents cleaning up the cloned repository. Installing it normally and removing the cloned directory will significantly reduce the Docker image size.
RUN git clone https://github.com/ModelTC/SpargeAttn.git --depth 1 && \n cd SpargeAttn && \n TORCH_CUDA_ARCH_LIST="8.0;8.6;8.9;9.0" pip install --no-cache-dir --no-build-isolation -v . && \n cd .. && \n rm -rf SpargeAttn
| FROM lightx2v/lightx2v:26062001-cu130 AS base | ||
| # ubuntu 24.04 python 3.12 torch 2.11 cuda 13.0 | ||
| FROM lightx2v/lightx2v:26070401-cu130 AS base | ||
| # ubuntu 24.04 python 3.11 torch 2.11 cuda 13.0 |
There was a problem hiding this comment.
The base image lightx2v/lightx2v:26070401-cu130 is built from Dockerfile_cu130, which uses Python 3.12 (as seen in its base image pytorch/pytorch:2.11.0-cuda13.0-cudnn9-devel and its flash_attn wheel). Changing this comment to python 3.11 is incorrect and misleading.
# ubuntu 24.04 python 3.12 torch 2.11 cuda 13.0
No description provided.