Possible untranslated Chinese DOCX content in the English split (Task 102 as a reproducible example)

[41ow1ives]

Summary

Hello, and thank you for releasing Workspace-Bench.

I noticed a possible language mismatch in the English dataset. Task 102 is marked as English ("language": "en"), and its task description, rubrics, manifest filenames, and expected output filename are all written in English. However, all four source DOCX files contain Chinese body text.

This affects both:

• Workspace-Bench Full: task_clean_en/102
• Workspace-Bench Lite: task_lite_clean_en/102

Task 102 is a concrete and reproducible example, but the issue may not be limited to this task. During manual spot checks, I found several other DOCX files in the English split whose filenames and surrounding metadata were in English while their document contents remained in Chinese. I have not performed an exhaustive audit, so I cannot yet quantify the full scope.

I also have not yet checked whether this issue affects file formats other than DOCX. If I identify additional affected tasks or file types, I will add the details to this issue.

Affected files in Task 102

The following four files in the English split contain Chinese body text:

• b26c1a99fb2df348_cost_reduction_and_efficiency_improvement_practical_plan_1.docx
• f58f30678f05f585_cost_reduction_and_efficiency_improvement_practical_plan_2.docx
• 9954618ef9744626_cost_reduction_and_efficiency_improvement_practical_plan_3.docx
• 94154069a2c066fb_cost_reduction_and_efficiency_improvement_practical_plan_4.docx

For example, the fourth document contains headings such as:

合理调度车辆减少车辆使用费
节约用水、用电

The corresponding metadata is explicitly marked as English:

• [Full metadata](https://huggingface.co/datasets/Workspace-Bench/Workspace-Bench/blob/main/task_clean_en/102/metadata.json)
• [Lite metadata](https://huggingface.co/datasets/Workspace-Bench/Workspace-Bench-Lite/blob/main/task_lite_clean_en/102/metadata.json)

I also compared the four English-split DOCX files with their counterparts in the Chinese split. Their document contents appear to be identical; only the filenames and surrounding task metadata have been translated into English.

How to reproduce

1. Open either English Task 102 directory linked above.
2. Confirm that metadata.json contains "language": "en" and an English task description.
3. Download and open any of the four DOCX input files.
4. Observe that the document body is written in Chinese.

Impact

If the English split is intended to provide an English-language evaluation, this may introduce an unintended language-proficiency confound:

• An English-only agent must understand Chinese to recover task-critical information.
• Performance may reflect Chinese document comprehension rather than the intended workspace-learning capabilities.
• Comparisons between models on the English split may be affected by differences in multilingual ability.
• Agents may produce incomplete or mixed-language deliverables despite correctly following the English task instruction.

Request for clarification and suggested fix

Could you please clarify whether source artifacts in the English split are expected to be fully English, or whether some tasks are intentionally cross-lingual?

If these Chinese DOCX contents are unintended, would it be possible to:

1. Translate or replace the affected DOCX files in both the Full and Lite English splits.
2. Preserve the original document structure and formatting where relevant.
3. Update any affected hashes, manifests, or workspace packages.
4. Run a broader audit of DOCX source artifacts under task_clean_en and task_lite_clean_en.

One possible audit would be to extract the body text from each English-split DOCX and identify files whose dominant language is Chinese or another unexpected language. Comparing English and Chinese split counterparts may also reveal cases where only the filename or metadata was localized.

If multilingual source documents are intentional, it would be helpful to document this explicitly or expose the source-document language separately—for example, through an input_languages field—so that users do not interpret "language": "en" as applying to both the task prompt and all of its source artifacts.

Expected outcome

It would be helpful if the maintainers could:

• Confirm the intended language policy for source artifacts in the English split.
• Audit the English Full and Lite splits for untranslated DOCX files.
• Replace affected documents or explicitly annotate intentionally cross-lingual tasks.
• Publish a list of affected tasks if the issue is broader than Task 102.

Thank you for reviewing this issue and for maintaining the benchmark.