Login

Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.online is one of the many independent Mastodon servers you can use to participate in the fediverse.
A newer server operated by the Mastodon gGmbH non-profit

Administered by:

Server stats:

10K
active users

mastodon.online: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.0-nightly.2025-04-15

#chumsky

0 posts0 participants0 posts today
Public
Shini92 :ablobblewobble:

To the #Rust #rustlang community: I once started to write a #compiler / #parser with #nom #crate. However I had some struggles with it of how to provide **multiple** errors with line+column indicators for a parsed context.
Now I read a tiny bit about #syn and #chumsky 👀
Are they the right crates for me? Are there others?
I do not parse rust-code, but a completely custom language (similar to pugjs)

(Boost for reach 🙏)

Public
Jan :rust: :ferris:

Sometimes, when programming in #Rust, I'm wondering whether there is a little Ferris in my computer whispering to all those 0's and 1's to behave correctly. :ferris:

Otherwise I can't describe the following experience:
1. Rewrite parts of #parsing a programming language from #nom to #chumsky (the parts that are most interleaved with each other => indirect recursion over multiple non-terminals!).
2. Run all affected tests
3. See all of them pass _on the first try!_

1/2

Terminal showing 32 tests pass when parsing conditional expressions. The exact output is: test parsers::conditional_expression::tests::or_and_expr ... ok test parsers::conditional_expression::tests::simple_expr_column ... ok test parsers::conditional_expression::tests::simple_expr_literal_double_quoted ... ok test parsers::conditional_expression::tests::simple_expr_literal_single_quoted ... ok test parsers::conditional_expression::tests::and_expr_parenthesized_or ... ok test parsers::conditional_expression::tests::variable_expr_not ... ok test parsers::conditional_expression::tests::or_or_and_expr ... ok test parsers::conditional_expression::tests::variable_expr_not_parens ... ok test parsers::conditional_expression::tests::variable_expr_not_parens_not ... ok test parsers::conditional_expression::tests::variable_expr_parens ... ok test parsers::conditional_expression::tests::logical_comparison_mixed_deep_parenthesized_expr ... ok test result: ok. 32 passed; 0 failed; 0 ignored; 0 measured; 173 filtered out; finished in 7.71s
Terminal showing 7 tests pass when parsing ternary expressions. The exact output is: running 7 tests test parsers::ternary_expression::tests::ternary_expr_not_parenthesized_comparison_op ... ok test parsers::ternary_expression::tests::ternary_expr_deep_else ... ok test parsers::ternary_expression::tests::ternary_expr_deep_ifs ... ok test parsers::ternary_expression::tests::ternary_expr_simple ... ok test parsers::ternary_expression::tests::ternary_expr_not_parenthesized_not_variable ... ok test parsers::ternary_expression::tests::ternary_expr_parenthesized ... ok test parsers::ternary_expression::tests::ternary_expr_test_expr_is_parenthesized_ternary_expr ... ok test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 198 filtered out; finished in 3.92s
Terminal showing 8 tests pass when parsing eval_name. The exact output is: running 8 tests test parsers::eval_name::tests::eval_name_qualifier ... ok test parsers::eval_name::tests::eval_name_looks_like_new_but_is_normal_identifier ... ok test parsers::eval_name::tests::eval_name_new ... ok test parsers::eval_name::tests::eval_name_qualifier_dblcolon ... ok test parsers::eval_name::tests::eval_name_std_id ... ok test parsers::eval_name::tests::eval_name_std_id_dblcolon_std_id ... ok test parsers::eval_name::tests::eval_name_super ... ok test parsers::eval_name::tests::eval_name_super_case_insensitive ... ok test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 197 filtered out; finished in 0.43s
Terminal showing 10 tests pass when parsing eval expressions. The exact output is: running 10 tests test parsers::eval::tests::eval_function_args_non ... ok test parsers::eval::tests::eval_new_method_args_simple ... ok test parsers::eval::tests::eval_function_args_simple ... ok test parsers::eval::tests::eval_qualifier_method_chained ... ok test parsers::eval::tests::eval_function_args_nested ... ok test parsers::eval::tests::eval_qualifier_method_args_simple ... ok test parsers::eval::tests::eval_qualifier_with_table_map_method ... ok test parsers::eval::tests::eval_static_method_args_simple ... ok test parsers::eval::tests::eval_qualifier_method_chained_with_args ... ok test parsers::eval::tests::eval_function_args_deeply_nested ... ok test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 195 filtered out; finished in 18.55s
#RustLang#Magic
Continued thread
Public
Jan :rust: :ferris:

So what should this all tell you? It should tell you that:
- #Rust enables fearless prototyping, refactoring and maintenance of software
- you shouldn't be afraid of stepping into the unknown
- despite seeming very scary at first, #chumsky is more approachable than you might think - thank you to chumsky contributors for such an amazing crate!

10/11

#RustLang
Public
Jan :rust: :ferris:

Many people say that #Rust is very hard to #prototype with or to #refactor. This couldn't be further from the truth! It is the exact opposite!

Let me share with you one of the most profound experiences I had with #RustLang on a casual sunday - a thread 🧵

I'm currently rewriting my #transpiler from #nom to #chumsky and until now everything has turned out great so far, until I've hit the following road block:
Implementing parsers by using #parser functions that have indirect #recursion.

1/11

Public
Jan :rust: :ferris:

The #Rust #parser combinator experience:

1. Writing multiple parser combinator functions ✅
2. The most "higher up" parser fails ❌
3. Investigate reason 🔍
4. Find out it's a _whitespace_ char that has been parsed too eagerly 💡
5. Fix eagerly parsed whitespace 🔧
6. All tests pass ✅
7. Rinse and repeat ➰

In all seriousness: 95% of errors I have are due to me parsing whitespace too eagerly (this is good!).

I'm using #chumsky, btw (beautifully designed parser combinator crate!).

#RustLang
Continued thread
Public
Jan :rust: :ferris:

Sometimes, in order to stay flexible and fit, you need to do some (type) gymnastics 🤸

Slowly getting the hang of #chumsky #parser combinator #crate 🤓

One concept (among others!) I haven't learned yet, but _will_ run into, is #recursion in chumsky. I know, there are good docs on it - it's just something I haven't gotten to yet.

All in all, I'm very happy with chumsky so far. Kudos to its maintainers! 🙂 🎩

fn column_function_chumsky_helper< 'a, I: Clone + AsRef<str> + ValueInput<'a>, F: Fn(String) -> FunctionExpression + Clone, >( keyword: I, fn_expr_out: F, ) -> impl Parser<'a, &'a str, FunctionExpression, extra::Err<Rich<'a, char>>> + Clone { text::ascii::keyword(keyword) .ignore_then(function_arg_chumsky().delimited_by(just('(').padded(), just(')').padded())) .map(fn_expr_out) } pub fn column_function_chumsky<'a>( ) -> impl Parser<'a, &'a str, FunctionExpression, extra::Err<Rich<'a, char>>> + Clone { choice(( column_function_chumsky_helper("count", FunctionExpression::Count), column_function_chumsky_helper("avg", FunctionExpression::Avg), column_function_chumsky_helper("sum", FunctionExpression::Sum), column_function_chumsky_helper("minOf", FunctionExpression::MinOf), column_function_chumsky_helper("maxOf", FunctionExpression::MaxOf), )) }
#Rust#RustLang
Public
Jan :rust: :ferris:

Currently in the process of rewriting my #transpiler from #nom v4.2 to #chumsky v1.0.0-alpha.6 🤓

It is a lot of fun so far, but I have to say these type signatures are _wild_!😄

I'm still struggling with it far more than I'd like, but I guess it is just a matter of time until intuition kicks in and it will become more and more natural.

Exciting project ahead!🙂

I'm even able to parse string literals _with escape sequences_ - something I haven't even achieved with nom!

Screenshot of my code editor, which is split in half horizontally. Code for parsing literal strings for my transpiler project with the chumsky parser combinator crate is shown at the top. The second half of the screen shows the terminal output after running the tests (all tests pass). What now follows is the exact code and output of the terminal: pub fn literal_chumsky<'a>( ) -> impl Parser<'a, &'a str, Constant, extra::Err<Simple<'a, char>>> + Clone { fn _literal<'a>( enclosing_char: char, ) -> impl Parser<'a, &'a str, String, extra::Err<Simple<'a, char>>> + Clone { let escape = just(['\\', enclosing_char]).to(enclosing_char); none_of::<_, _, extra::Err<Simple<char>>>(enclosing_char) .and_is(escape.not()) .or(escape) .repeated() .collect::<String>() .padded_by(just(enclosing_char)) } _literal('"').or(_literal('\'')).map(Constant::String) } running 6 tests test parsers::common::tests::column_idx_overflow_error ... ok test parsers::common::tests::literal_with_escape ... ok test parsers::common::tests::literal_without_escape ... ok test parsers::common::tests::column_idx_max_u32_works ... ok test parsers::common::tests::xpp_identifier ... ok test parsers::common::tests::xpp_identifier_with_additional_content_at_end ... ok test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 191 filtered out; finished in 0.00s
#Rust#RustLang
Public
Jan :rust: :ferris:

Uh, ohhh... I think it's time for me to migrate away from #nom v4.2 😮

Yeah, I know, I've procrastinated on this a lot. This will probably be a lot of work and "slow me down" for a bit. 😪 On the upside, though: I can correct all my mistakes along the way (like having spans).

I'll probably migrate to #chumsky, but #winnow also looks really nice. 🙂

chumsky:
github.com/zesterer/chumsky

winnow:
github.com/winnow-rs/winnow

Output of my terminal when running unit tests of `selecuery` (a transpiler/compiler). It says: "warning: the following packages contain code that will be rejected by a future version of Rust: nom v4.2.3" Then, on the next line: "note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`"
#Rust#RustLang#selecuery
Public *
Jan :rust: :ferris:

winnow - Making parsing a breeze (by epage and contributors):

github.com/winnow-rs/winnow

winnow is a parser combinator library written in Rust that started as a fork of #nom.

Not sure yet, if I should migrate from nom 4.* to #winnow or #chumsky. I really do like the good error recovery and parsing capabilities for PEGs of chumsky.🥰

Making parsing a breeze. Contribute to winnow-rs/winnow development by creating an account on GitHub.
GitHubGitHub - winnow-rs/winnow: Making parsing a breezeMaking parsing a breeze. Contribute to winnow-rs/winnow development by creating an account on GitHub.
#Rust#RustLang#Parser
ExploreLive feeds
About