Improve Falcon parser #1079

Activity

1. sirosen commented on Jul 22, 2026

   

   sirosen

   on Jul 22, 2026

   Collaborator

   More actions

   Meeting everyone at that roundtable was also a highlight for me! I'll be working in the coming weeks on finding ways to keep that conversation going.

   @lafrech, @sloria, as a quick explanation, Vytas and I met this past weekend at EuroPython; he's one of the maintainers of Falcon.
   I'll be sending some emails when I get home, including ones to each of you, to which will make some sense of this subthread. 😄

   IIRC I offered to help to improve Falcon "parser", but now that I've opened the code, it is not as trivial as I thought.

   I thought I was going to try to do it and run it past you, but I can't pretend to remember perfectly.
   I'll look at the draft PR within the next couple of days and we can discuss details there.

   We were preparing to release a v9.0 of webargs in the fall, but it got delayed. That may turn out to be advantageous for us, as it gives us some freedom to make breaking changes as needed.

   🚀React with 🚀1Reacted by Inessa Pawson
2. sloria commented on Jul 23, 2026

   

   sloria

   on Jul 23, 2026

   Member

   More actions

   very cool! looking forward to hearing about it @sirosen . and thanks for offering to help @vytas7
3. sirosen commented on Aug 1, 2026

   

   sirosen

   on Aug 1, 2026

   Collaborator

   More actions

   I finally took some time to start looking through things here! (got swamped with lots of priorities when I came back from EuroPython 😅 )

   Also, sorry, this post is quite long because it's hard to condense the information I want to share! 🥲

   Item 1: URL-encoded form handling

   Do we want to DRY URL-encoded forms with Falcon's media handling? Do we need to keep the parse_form_body(...) helper for backwards compatibility, or is it considered private? Do we want to silence Unicode errors as it does?

   I think we should rename it, in our v9.0 release (dev branch). We're not intentionally exposing it, so let's make it _parse_form_body. My conclusion is that we need to keep it. We should realign it with Falcon, which I think means adding the csv=... arg and not much else?

   I reach this after realizing that Falcon and webargs have APIs which do not match, and the cost to our users of aligning them outweighs the cost to us of doing some extra work.

   webargs is trying to expose the following methods:

   • FalconParser.load_querystring: the query parameters as a MultiDictProxy¹
   • FalconParser.load_form: the form body as a MultiDictProxy
   • FalconParser.load_json_or_form: inherited from core.Parser; wraps load_form() to try JSON and failover to a form body

   In Falcon, when a form body is parsed, it is combined with the query params.

   FalconParser.load_querystring is currently using req.params, so that seems... wrong. It will include things from a form-post body as well, but we called it "querystring" to match our other parsers.

   If we wanted to true up our APIs, we'd make all three of those webargs methods I mentioned -- including the inherited one, override it -- raise NotImplementedError because they don't match Falcon. We'd provide load_params instead, for handling falcon.Request.params. I don't really want to do this: the code we have to copy is not so big, and I think the user-facing breakage is confusing and surprising.

   That analysis gives me a second conclusion: FalconParser.load_querystring() should possibly start emitting a warning. I'll file it as an issue for us to assess this during the webargs v9.x lifecycle.² We might deprecate/remove it, or if we're okay with the "lie" in the method name we'll keep it but add a doc warning or similar.

   Item 2: async support

   How to load media/request body in the async case? Is it properly supported for AIOHTTP?

   My first inclination is not to add anything or do anything until a user comes to us with a use-case.

   We have a level of orchestration one layer higher which handles choosing sync vs async codepaths. If we define async def load_media where we currently have def load_media, everything would work, but it would break the sync case... which is no good.

   We could easily add a separate parser, AsyncFalconParser or a separate "location" in webargs: async def load_media_async. But until I'm talking to a user who wants this, it's hard to be confident about the API.

   Item 3: adding a recipe

   Add a recipe how to integrate this in middleware (in possible).

   I'd welcome such an addition, in docs/framework_support.rst if it's short!
   You'll see a minimal Falcon usage example there, and I wouldn't mind a slightly larger one.

   If it's much bigger, we can put it in our examples dir, perhaps?

   Footnotes

   1. MultiDictProxy is part of the webargs problem-space. We wrap dicts and multidicts, to handle get() vs get_list()/get_all() semantics smoothly, when combined with marshmallow schemas. You can mostly ignore and treat it as "special marshmallow-facing dict". ↩

   2. I won't file it right away, in case we reach a different conclusion. ↩
4. vytas7 commented on Aug 1, 2026

   

   vytas7

   on Aug 1, 2026

   Last edited by vytas7

   Author

   More actions

   Thanks for the detailed feedback @sirosen!

   Just to clarify Falcon's parsing of URL-encoded forms: no, the form is not automatically combined with req.params. This only happens if the deprecated req_options.auto_parse_form_urlencoded option is used. In fact, this option and the code block you referenced will be removed in the next major version (we don't bump these too often though, so maybe in ~3 years).

   The canonical way is to simply use req.get_media() to parse the request body, that includes URL-encoded forms too. The URL-encoded form media handler is installed by default.

   So what load_querystring does is not incorrect per se.

   Regarding these 3 load_* methods in general, is that what defines a webargs "parser" interface?

   Edited/added ⬇️

   if we are concerned about FalconParser.load_querystring being incorrect when this option is used, we could emit a warning specifically in the case req.options.auto_parse_form_urlencoded is enabled.

   In fact, when the user tries to enable this behaviour, Falcon already emits a warning on its own:

   >>> import falcon.testing
   >>> req = falcon.testing.create_req()
   >>> req.options.auto_parse_form_urlencoded
   False
   >>> req.options.auto_parse_form_urlencoded = True
   <...>/falcon/falcon/request.py:2820: DeprecatedWarning: The RequestOptions.auto_parse_form_urlencoded option is deprecated. Please use Request.get_media() to consume the submitted URL-encoded form instead.
     warnings.warn(

   👍React with 👍1Reacted by Stephen Rosen
5. sirosen commented on Aug 1, 2026

   

   sirosen

   on Aug 1, 2026

   Collaborator

   More actions

   Ah! I even read straight past the "auto" option, but didn't understand what I was seeing.
   I need to take some time with this, in that case. I still think there's a mismatch, but it's not quite the one I described.

   We list "locations" for getting data: query params, headers, etc. And for the body, we have multiple "locations" which are actually the parsing methods: "json", "form", etc. For the Falcon parser, we have a location named "media", which calls get_media().

   One option: we could make load_form check the media type, and then call get_media if that passes. We'd either return our missing sentinel or raise an error if the check fails (I'm not sure offhand). Does that sound like a reasonable way to wrap get_media()?
6. vytas7 commented on Aug 2, 2026

   

   vytas7

   on Aug 2, 2026

   Author

   More actions

   One option: we could make load_form check the media type, and then call get_media if that passes. We'd either return our missing sentinel or raise an error if the check fails (I'm not sure offhand). Does that sound like a reasonable way to wrap get_media()?

   Aye, that's exactly what I was suggesting.

   Just a couple more details, and I can go on to materialize this in my PR:

   1. Unlike JSON, Falcon treats an empty body as valid URL-encoded data (deserializes to {}). Is that in line with webargs' behaviour?
   2. Regarding other media types passed to load_form(), it's your call 🙂 I think there are two branches here though: one, a multipart form is submitted -- Falcon fully supports parsing it, but unlike other frameworks, the user has to iterate the form (nothing is automatically spooled in memory or temporary files behind your back) -- since this is not supported by webargs yet, we can raise an error; two, the payload is any other non-form media -- here it's your call.
7. sirosen commented on Aug 13, 2026

   

   sirosen

   on Aug 13, 2026

   Collaborator

   More actions

   1. Unlike JSON, Falcon treats an empty body as valid URL-encoded data (deserializes to {}). Is that in line with webargs' behaviour?

   Yep! (I also think that's the most appropriate result in the abstract for an empty string, in the abstract.)

   2. Regarding other media types passed to load_form(), it's your call 🙂 I think there are two branches here though: one, a multipart form is submitted -- Falcon fully supports parsing it, but unlike other frameworks, the user has to iterate the form (nothing is automatically spooled in memory or temporary files behind your back) -- since this is not supported by webargs yet, we can raise an error; two, the payload is any other non-form media -- here it's your call.

   I've looked through the surrounding codepaths in the base parser, and I think we should

   • Error on multipart form -- we can add support later, but I'd rather do that with a clear test case to think about.
     • Because we'll release this all as part of v9.0, it's okay if it breaks some current usage which worked by accident.
   • Return missing if the payload is non-form media.
     • Although I am not certain this is really what we ought to do, it's how we behave in some similar contexts (like JSON data with no Content-Type). Being consistent for now is good, and we can change it in a future major version if we find in practice that there are issues.

   ---

   Thanks so much for helping us with this, by the way! It's a huge lift to have the expertise of someone who works on the frameworks. (Plus it's nice to stay in touch. 🫶 )