Parsing primitives

These are the lowest level building blocks for creating parsers.

parsy.string(expected_string, transform=None)[source]

Returns a parser that expects the expected_string and produces that string value.

Optionally, a transform function can be passed, which will be used on both the expected string and tested string. This allows things like case insensitive matches to be done. This function must not change the length of the string (as determined by len). The returned value of the parser will always be expected_string in its un-transformed state.

>>> parser = string("Hello", transform=lambda s: s.upper())
>>> parser.parse("Hello")
'Hello'
>>> parser.parse("hello")
'Hello'
>>> parser.parse("HELLO")
'Hello'

Changed in version 1.2: Added transform argument.

parsy.regex(exp, flags=0, group=0)[source]

Returns a parser that expects the given exp, and produces the matched string. exp can be a compiled regular expression, or a string which will be compiled with the given flags.

Optionally, accepts group, which is passed to re.Match.group to return the text from a capturing group in the regex instead of the entire match.

Using a regex parser for small building blocks, instead of building up parsers from primitives like string(), test_char() and Parser.times() combinators etc., can have several advantages, including:

  • It can be more succinct e.g. compare:

    >>> (string('a') | string('b')).times(1, 4)
    >>> regex(r'[ab]{1,4}')
    
  • It can return the entire matched string as a single item, so you don’t need to use Parser.concat().

  • It can return a part of the matched string using a capturing group from the regex, so you don’t need to split the string yourself.

    You can use named or numbered groups, just like with re.Match.group. Tuples also work, and return the captured text from multiple groups.

    >>> regex(r'([0-9]{4})-([0-9]{2})', group=1).parse('2020-03')
    '2020'
    >>> regex(r'(?P<year>[0-9]{4})-(?P<month>[0-9]{2})', group='month').parse('2020-03')
    '03'
    >>> regex(r'([0-9]{4})-([0-9]{2})', group=(1,2)).parse('2020-03')
    ('2020', '03')
    
  • It can be much faster.

parsy.test_char(func, description)[source]

Returns a parser that tests a single character with the callable func. If func returns True, the parse succeeds, otherwise the parse fails with the description description.

>>> ascii = test_char(lambda c: ord(c) < 128,
...                   'ascii character')
>>> ascii.parse('A')
'A'
parsy.test_item(func, description)[source]

Returns a parser that tests a single item from the list of items being consumed, using the callable func. If func returns True, the parse succeeds, otherwise the parse fails with the description description.

If you are parsing a string, i.e. a list of characters, you can use test_char() instead. (In fact the implementations are identical, these functions are aliases for the sake of clear code).

>>> numeric = test_item(str.isnumeric, 'numeric')
>>> numeric.many().parse(['123', '456'])
['123', '456']
parsy.char_from(characters)[source]

Accepts a string and returns a parser that matches and returns one character from the string.

>>> char_from('abc').parse('a')
'a'
parsy.string_from(*strings, transform=None)[source]

Accepts a sequence of strings as positional arguments, and returns a parser that matches and returns one string from the list. The list is first sorted in descending length order, so that overlapping strings are handled correctly by checking the longest one first.

>>> string_from('y', 'yes').parse('yes')
'yes'

Optionally accepts transform, which is passed to string() (see the documentation there).

Changed in version 1.2: Added transform argument.

parsy.match_item(item, description=None)[source]

Returns a parser that tests the next item (or character) from the stream (or string) for equality against the provided item. Optionally a string description can be passed.

Parsing a string:

>>> letter_A = match_item('A')
>>> letter_A.parse_partial('ABC')
('A', 'BC')

Parsing a list of tokens:

>>> hello = match_item('hello')
>>> hello.parse_partial(['hello', 'how', 'are', 'you'])
('hello', ['how', 'are', 'you'])
parsy.eof[source]

A parser that only succeeds if the end of the stream has been reached.

>>> eof.parse_partial("")
(None, '')
>>> eof.parse_partial("123")
Traceback (most recent call last):
   ...
parsy.ParseError: expected 'EOF' at 0:0
parsy.success(val)[source]

Returns a parser that does not consume any of the stream, but produces val.

parsy.fail(expected)[source]

Returns a parser that always fails with the provided error message.

parsy.from_enum(enum_cls, transform=None)[source]

Given a class that is an enum.Enum class, returns a parser that will parse the values (or the string representations of the values) and return the corresponding enum item.

>>> from enum import Enum
>>> class Pet(Enum):
...     CAT = "cat"
...     DOG = "dog"
>>> pet = from_enum(Pet)
>>> pet.parse("cat")
<Pet.CAT: 'cat'>

str is first run on the values (for the case of values that are integers etc.) to create the strings which are turned into parsers using string().

If transform is provided, it is passed to string() when creating the parser (allowing for things like case insensitive parsing).

parsy.peek(parser)[source]

Returns a lookahead parser that parses the input stream without consuming chars.

Pre-built parsers

Some common, pre-built parsers (all of these are Parser objects created using the primitives above):

parsy.any_char

A parser that matches any single character.

parsy.whitespace

A parser that matches and returns one or more whitespace characters.

parsy.letter

A parser that matches and returns a single letter, as defined by str.isalpha.

parsy.digit

A parser that matches and returns a single digit, as defined by str.isdigit. Note that this includes various unicode characters outside of the normal 0-9 range, such as ¹²³.

parsy.decimal_digit

A parser that matches and returns a single decimal digit, one of “0123456789”.

parsy.line_info

A parser that consumes no input and always just returns the current line information, a tuple of (line, column), zero-indexed, where lines are terminated by \n. This is normally useful when wanting to build more debugging information into parse failure error messages.

parsy.index

A parser that consumes no input and always just returns the current stream index. This is normally useful when wanting to build more debugging information into parse failure error messages.

Forward declarations

class parsy.forward_declaration[source]

When defining parsers for a recursive grammar, you may run into NameError problems with a naive approach, because you can’t refer to a Python object before you have defined it. In this case, forward_declaration can be useful.

Say we want to be able to parse an s-expression like syntax which uses parenthesis for grouping items into a tree structure, like the following:

(0 1 (2 3) (4 5 6) 7 8)

A naive approach would be:

simple = regex('[0-9]+').map(int)
group = string('(') >> expr.sep_by(string(' ')) << string(')')
expr = simple | group

The problem is that the second line will get a NameError because expr is not defined yet, and we’ll have the same problem if we put the expr definition first.

We can solve it like this:

from parsy import forward_declaration, regex, string

expr = forward_declaration()
simple = regex('[0-9]+').map(int)
group = string('(') >> expr.sep_by(string(' ')) << string(')')
expr.become(simple | group)

You must use .become() method exactly once before attempting to use the parser.

An alternative to this is to use generate as described in Implementing recursive definitions.