fix: partial write error handling - #239
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors write error handling to better distinguish partial-write errors from generic write failures by parsing JSON error bodies and conditionally raising InfluxDBPartialWriteError only when accept_partial is enabled and the response matches the expected “rejected rows” list format.
Changes:
- Update write exception translation to parse JSON bodies and format per-line rejection details for partial writes.
- Adjust sync REST client response decoding to normalize empty bodies to
None. - Expand and update tests to cover partial-write detection, message formatting, and fallback behaviors.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_write_api.py |
Adds broader unit coverage for write-error classification and fallback messaging; updates existing error-translation tests. |
tests/test_influxdb_client_3_integration.py |
Loosens assertion on v3 error message content and narrows the tested accept_partial configuration. |
influxdb_client_3/write_client/client/write_api.py |
Implements new JSON parsing and partial-write detection/formatting in _translate_write_exception. |
influxdb_client_3/write_client/_sync/rest_client.py |
Avoids decoding empty bodies and normalizes empty data to None. |
influxdb_client_3/exceptions/exceptions.py |
Changes exception message handling and updates partial-write error types/constructors. |
Suppressed comments (1)
influxdb_client_3/exceptions/exceptions.py:69
- These PEP 604 union annotations (
int | None, etc.) will fail to parse on Python <3.10. UseOptional[...](withOptionalimported fromtyping) to preserve compatibility.
class InfluxDBPartialWriteLineError:
line_number: int | None
error_message: str | None
original_line: str | None
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def __init__(self, response: HTTPResponse = None, message: str = None): | ||
| """Initialize the InfluxDBError handler.""" | ||
| if response is not None: | ||
| self.response = response | ||
| self.message = self._get_message(response) | ||
| self.message = message | ||
| self.retry_after = response.getheader('Retry-After') | ||
| else: | ||
| self.response = None | ||
| self.message = message or 'no response' | ||
| self.retry_after = None | ||
| super().__init__(self.message) |
0e798b1 to
3352e90
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
influxdb_client_3/exceptions/exceptions.py:5
- Python 3.9 is still in the CI matrix, but this module now uses PEP 604 union syntax elsewhere (
int | None), so you’ll needOptionalavailable for 3.9-compatible type annotations.
from typing import List
influxdb_client_3/exceptions/exceptions.py:69
int | None/str | Nonetype syntax is a Python 3.10+ feature and will raise a SyntaxError on Python 3.9 (which is still in.github/workflows/pylint.yml). UseOptional[...]instead for compatibility.
class InfluxDBPartialWriteLineError:
line_number: int | None
error_message: str | None
original_line: str | None
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #239 +/- ##
==========================================
- Coverage 87.43% 84.31% -3.13%
==========================================
Files 28 28
Lines 2030 2015 -15
==========================================
- Hits 1775 1699 -76
- Misses 255 316 +61 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1ea4183 to
1c5111d
Compare
28581e6 to
4f769ef
Compare
bednar
left a comment
There was a problem hiding this comment.
I found five issues that need to be addressed before approval.
dc587df to
90321e4
Compare
f5f8ace to
dc4f14d
Compare
dc4f14d to
98610a0
Compare
b0f6f95 to
7d5c7e6
Compare
7d5c7e6 to
2919ab7
Compare
Closes #
Proposed Changes
The current exception classes hierarchy are
InfluxDBError->InfluxDBPartialWriteErrorandApiExceptionInfluxDBErrorthere is an important function_get_message(self, response), this function will run everytime a subclass ofInfluxDBErroris initialized, inside this function there is a quite heavy function will run, that is_parse_partial_write_line_error_info(data), so when a partial write error occurs_get_message(self, response)will be called at least twice.Some issues:
InfluxDBErrorwill have the same way of parsing the error messages like we do in_get_message(self, response)function is not correct... I think.We can try to make everything work more efficiently as possible with the current exception classes implementation, but I still feel It very wrong on the architecture and design perspective.
What I'm trying to do right now is moving all logic inside exception classes to WriteApi class (or wherever than inside exception classes) and making them as lightweight as possible. They should only carry information about the errors;
Checklist