Step 2 of 6 for the survey2ddi retirement. Plan: HANDOVER_SURVEY2DDI.md. Independent of the xlsform2ddi --data issue; both are needed before survey2ddi's CLIs can be deprecated.
The gap
lstsvToDdiXml emits metadata. buildDataCsv (#8) emits a data CSV from submissions keyed by bare question name or group/name path. Neither shape is what a LimeSurvey response export produces, so there is no LimeSurvey full-mode path in this library.
limesurvey2ddi/transform.py in CorrelAid/survey2ddi does the missing step — normalize_responses(variables, rows):
- Bracket subkeys. LimeSurvey exports a
select_multiple as one column per subquestion, keyed with a bracket suffix. _match_choice(subkey, choices) maps that suffix back to the parsed choice code so it can line up with the <name>_<choice> DDI variable.
- The underscore quirk.
_norm(name) strips underscores and lowercases, "mirrors LimeSurvey's export behaviour" — LimeSurvey mangles field names on export, so matching must be done on the normalized form.
Both are LimeSurvey-specific and belong nowhere near the Kobo path.
What to build
src/pipelines/lstsv2ddi/data.ts, mirroring how xlsform2ddi/data.ts sits beside variables.ts:
lstsvToDataCsv(tsv: string, responses: Record<string, unknown>[], options?): string
Simplest honest shape: a key-normalizing adapter that re-keys LimeSurvey rows onto DDI variable names, then delegates to the existing buildDataCsv — the RFC 4180 writing and the column plan are already correct and must not be duplicated. Do not widen readCell in xlsform2ddi/data.ts with LimeSurvey heuristics; the two platforms' quirks stay separate.
Then add --data / --data-out to the lstsv2ddi CLI command, same flags and semantics as the xlsform2ddi issue.
Verification
- Fixture-based unit test: a LimeSurvey response export (bracketed multi columns, underscore-mangled names) → correct
0/1 binaries and correct scalar columns
- Header order equals
<var name=""> order of lstsvToDdiXml on the same TSV
- Byte-compare against Python
normalize_responses + build_data_csv on one fixture, accounting for the known column-order divergence in the Python emitter (documented in the handover doc)
tests/fixtures/lstsv/ in survey2ddi has three usable structure TSVs
Acceptance criteria
Step 2 of 6 for the survey2ddi retirement. Plan:
HANDOVER_SURVEY2DDI.md. Independent of thexlsform2ddi --dataissue; both are needed before survey2ddi's CLIs can be deprecated.The gap
lstsvToDdiXmlemits metadata.buildDataCsv(#8) emits a data CSV from submissions keyed by bare question name orgroup/namepath. Neither shape is what a LimeSurvey response export produces, so there is no LimeSurvey full-mode path in this library.limesurvey2ddi/transform.pyinCorrelAid/survey2ddidoes the missing step —normalize_responses(variables, rows):select_multipleas one column per subquestion, keyed with a bracket suffix._match_choice(subkey, choices)maps that suffix back to the parsed choice code so it can line up with the<name>_<choice>DDI variable._norm(name)strips underscores and lowercases, "mirrors LimeSurvey's export behaviour" — LimeSurvey mangles field names on export, so matching must be done on the normalized form.Both are LimeSurvey-specific and belong nowhere near the Kobo path.
What to build
src/pipelines/lstsv2ddi/data.ts, mirroring howxlsform2ddi/data.tssits besidevariables.ts:Simplest honest shape: a key-normalizing adapter that re-keys LimeSurvey rows onto DDI variable names, then delegates to the existing
buildDataCsv— the RFC 4180 writing and the column plan are already correct and must not be duplicated. Do not widenreadCellinxlsform2ddi/data.tswith LimeSurvey heuristics; the two platforms' quirks stay separate.Then add
--data/--data-outto thelstsv2ddiCLI command, same flags and semantics as thexlsform2ddiissue.Verification
0/1binaries and correct scalar columns<var name="">order oflstsvToDdiXmlon the same TSVnormalize_responses+build_data_csvon one fixture, accounting for the known column-order divergence in the Python emitter (documented in the handover doc)tests/fixtures/lstsv/in survey2ddi has three usable structure TSVsAcceptance criteria
_normnormalizationlstsv2ddi --datawrites a CSV whose header matches the XMLlstsv2ddi/, not in the shared emitterdata.test.tsstill green)