Struct icu::pattern::Parser

pub struct Parser<'p, P> { /* private fields */ }

Placeholder pattern parser.

The parser allows for handling flexible range of generic patterns with placeholders.

The Parser is generic over any placeholder which implements FromStr allowing the consumer to parse placeholder patterns such as “{0}, {1}”, “{date}, {time}” or any other. A placeholder must be enclosed in { and } characters in the input pattern string.

At the moment the parser is written as a custom fallible iterator.

✨ Enabled with the alloc Cargo feature.

Examples

use icu_pattern::{ParsedPatternItem, Parser, ParserOptions};

let input = "{0}, {1}";

let mut parser = Parser::new(input, ParserOptions::default());

let mut result = vec![];

while let Some(element) =
    parser.try_next().expect("Failed to advance iterator")
{
    result.push(element);
}

assert_eq!(
    result,
    &[
        ParsedPatternItem::Placeholder(0),
        ParsedPatternItem::Literal {
            content: ", ".into(),
            quoted: false
        },
        ParsedPatternItem::Placeholder(1),
    ]
);

Named placeholders

The parser is also capable of parsing different placeholder types such as strings.

Examples

use icu_pattern::{ParsedPatternItem, Parser, ParserOptions};

let input = "{start}, {end}";

let mut parser = Parser::new(input, ParserOptions::default());

let mut result = vec![];

while let Some(element) =
    parser.try_next().expect("Failed to advance iterator")
{
    result.push(element);
}

assert_eq!(
    result,
    &[
        ParsedPatternItem::Placeholder("start".to_owned()),
        ParsedPatternItem::Literal {
            content: ", ".into(),
            quoted: false
        },
        ParsedPatternItem::Placeholder("end".to_owned()),
    ]
);

Type parameters

• P: The type of the placeholder used as a key for the PlaceholderValueProvider.

Lifetimes

• p: The life time of an input string slice to be parsed.

Design Decisions

The parser is written in an intentionally generic way to enable use against wide range of potential placeholder pattern models and use cases.

Serveral design decisions have been made that the reader should be aware of when using the API.

Zero copy

The parser is intended for runtime use and is optimized for performance and low memory overhad.

Zero copy parsing is a model which allows the parser to produce tokens that are de-facto slices of the input without ever having to modify the input or copy from it.

In case of ICU patterns that decision brings a trade-off around handling of quoted literals. A parser that copies bytes from the input when generating the output can take a pattern literal that contains a quoted portion and concatenace the parts, effectively generating a single literal out of a series of syntactical literal quoted and unquoted nodes. A zero copy parser sacrifices that convenience for marginal performance gains.

The rationale for the decision is that many placeholder patterns do not contain ASCII letters and therefore can benefit from this design decision. Secondly, even in scenarios where ASCII letters, or other quoted literals, are used, the zero-copy design still maintains high performance, only increasing the number of tokens returned by the parser, but without increase to allocations.

Examples

use icu_pattern::{ParsedPatternItem, Parser, QuoteMode};

let input = "{0} 'and' {1}";

let mut parser = Parser::new(input, QuoteMode::QuotingSupported.into());

let mut result = vec![];

while let Some(element) =
    parser.try_next().expect("Failed to advance iterator")
{
    result.push(element);
}

assert_eq!(
    result,
    &[
        ParsedPatternItem::Placeholder(0),
        ParsedPatternItem::Literal {
            content: " ".into(),
            quoted: false
        },
        ParsedPatternItem::Literal {
            content: "and".into(),
            quoted: true
        },
        ParsedPatternItem::Literal {
            content: " ".into(),
            quoted: false
        },
        ParsedPatternItem::Placeholder(1),
    ]
);

Fallible Iterator

Rust providers a strong support for iterators and iterator combinators, which fits very well into the design of this parser/interpolator model.

Unfortunately, Rust iterators at the moment are infallible, while parsers are inhereantely fallible. As such, the decision has been made to design the API in line with what we hope will become a trait signature of a fallible iterator in the future, rather than implementing a reversed infallible iterator (where the Item would be Option<Result<Item>>).

That decision impacts the ergonomics of operating on the parser, on one hand making the fallible iteration more ergonomic, at a trade-off of losing access to the wide range of Rust iterator traits.

Generic Placeholder

To handle generic placeholder design, the only constrain necessary in the parser is that a placeholder must be parsed from a string slice. At the moment of writing, Rust is preparing to deprecate FromStr in favor of TryFrom<&str>. Among many benfits of such transition would be the auto-trait behavior of From and a TryFrom<&str> for &str allowing for placeholders to be &str themselves.

Unfortunately, at the moment TryFrom<&str> for usize is not implemented, which would impact the core use case of placeholder patterns.

In result, the decision has been made to use FromStr for the time being, until TryFrom<&str> gets implemented on all types that support FromStr.

Implementations

impl<'p, P> Parser<'p, P>

pub fn new(input: &'p str, options: ParserOptions) -> Parser<'p, P>

Creates a new Parser.

The allow_raw_letters controls whether the parser will support ASCII letters without quotes.

Examples

use icu_pattern::{Parser, ParserOptions};
let mut parser = Parser::<usize>::new("{0}, {1}", ParserOptions::default());

pub fn try_next( &mut self, ) -> Result<Option<ParsedPatternItem<'p, P>>, ParserError<<P as FromStr>::Err>>

where P: FromStr, <P as FromStr>::Err: Debug,

An iterator method that advances the iterator and returns the result of an attempt to parse the next token.

Examples

use icu_pattern::{ParsedPatternItem, Parser, ParserOptions};

let mut parser = Parser::<usize>::new("{0}, {1}", ParserOptions::default());

// A call to try_next() returns the next value…
assert_eq!(
    Ok(Some(ParsedPatternItem::Placeholder(0))),
    parser.try_next()
);
assert_eq!(
    Ok(Some(ParsedPatternItem::Literal {
        content: ", ".into(),
        quoted: false
    })),
    parser.try_next()
);
assert_eq!(
    Ok(Some(ParsedPatternItem::Placeholder(1))),
    parser.try_next()
);

// … and then `None` once it's over.
assert_eq!(Ok(None), parser.try_next());

pub fn try_collect_into_vec( self, ) -> Result<Vec<ParsedPatternItem<'p, P>>, ParserError<<P as FromStr>::Err>>

where P: FromStr, <P as FromStr>::Err: Debug,

Mutates this parser and collects all ParsedPatternItems into a vector.

Trait Implementations

impl<'p, P> Debug for Parser<'p, P>

where P: Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'a, K> Iterator for Parser<'a, K>

where K: FromStr, <K as FromStr>::Err: Debug,

type Item = Result<PatternItemCow<'a, K>, PatternError>

The type of the elements being iterated over.

fn next(&mut self) -> Option<<Parser<'a, K> as Iterator>::Item>

Advances the iterator and returns the next value.

fn next_chunk<const N: usize>( &mut self, ) -> Result<[Self::Item; N], IntoIter<Self::Item, N>>

where Self: Sized,

🔬This is a nightly-only experimental API. (iter_next_chunk)

Advances the iterator and returns an array containing the next N values.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the iterator.

fn count(self) -> usize

where Self: Sized,

Consumes the iterator, counting the number of iterations and returning it.

fn last(self) -> Option<Self::Item>

where Self: Sized,

Consumes the iterator, returning the last element.

fn advance_by(&mut self, n: usize) -> Result<(), NonZero<usize>>

🔬This is a nightly-only experimental API. (iter_advance_by)

Advances the iterator by n elements.

fn nth(&mut self, n: usize) -> Option<Self::Item>

Returns the nth element of the iterator.

fn step_by(self, step: usize) -> StepBy<Self>

where Self: Sized,

Creates an iterator starting at the same point, but stepping by the given amount at each iteration.

fn chain<U>(self, other: U) -> Chain<Self, <U as IntoIterator>::IntoIter>

where Self: Sized, U: IntoIterator<Item = Self::Item>,

Takes two iterators and creates a new iterator over both in sequence.

fn zip<U>(self, other: U) -> Zip<Self, <U as IntoIterator>::IntoIter>

where Self: Sized, U: IntoIterator,

‘Zips up’ two iterators into a single iterator of pairs.

fn intersperse_with<G>(self, separator: G) -> IntersperseWith<Self, G>

where Self: Sized, G: FnMut() -> Self::Item,

🔬This is a nightly-only experimental API. (iter_intersperse)

Creates a new iterator which places an item generated by separator between adjacent items of the original iterator.

fn map<B, F>(self, f: F) -> Map<Self, F>

where Self: Sized, F: FnMut(Self::Item) -> B,

Takes a closure and creates an iterator which calls that closure on each element.

fn for_each<F>(self, f: F)

where Self: Sized, F: FnMut(Self::Item),

Calls a closure on each element of an iterator.

fn filter<P>(self, predicate: P) -> Filter<Self, P>

where Self: Sized, P: FnMut(&Self::Item) -> bool,

Creates an iterator which uses a closure to determine if an element should be yielded.

fn filter_map<B, F>(self, f: F) -> FilterMap<Self, F>

where Self: Sized, F: FnMut(Self::Item) -> Option<B>,

Creates an iterator that both filters and maps.

fn enumerate(self) -> Enumerate<Self>

where Self: Sized,

Creates an iterator which gives the current iteration count as well as the next value.

fn peekable(self) -> Peekable<Self>

where Self: Sized,

Creates an iterator which can use the peek and peek_mut methods to look at the next element of the iterator without consuming it. See their documentation for more information.

fn skip_while<P>(self, predicate: P) -> SkipWhile<Self, P>

where Self: Sized, P: FnMut(&Self::Item) -> bool,

Creates an iterator that skips elements based on a predicate.

fn take_while<P>(self, predicate: P) -> TakeWhile<Self, P>

where Self: Sized, P: FnMut(&Self::Item) -> bool,

Creates an iterator that yields elements based on a predicate.

fn map_while<B, P>(self, predicate: P) -> MapWhile<Self, P>

where Self: Sized, P: FnMut(Self::Item) -> Option<B>,

Creates an iterator that both yields elements based on a predicate and maps.

fn skip(self, n: usize) -> Skip<Self>

where Self: Sized,

Creates an iterator that skips the first n elements.

fn take(self, n: usize) -> Take<Self>

where Self: Sized,

Creates an iterator that yields the first n elements, or fewer if the underlying iterator ends sooner.

fn scan<St, B, F>(self, initial_state: St, f: F) -> Scan<Self, St, F>

where Self: Sized, F: FnMut(&mut St, Self::Item) -> Option<B>,

An iterator adapter which, like fold, holds internal state, but unlike fold, produces a new iterator.

fn flat_map<U, F>(self, f: F) -> FlatMap<Self, U, F>

where Self: Sized, U: IntoIterator, F: FnMut(Self::Item) -> U,

Creates an iterator that works like map, but flattens nested structure.

fn map_windows<F, R, const N: usize>(self, f: F) -> MapWindows<Self, F, N>

where Self: Sized, F: FnMut(&[Self::Item; N]) -> R,

🔬This is a nightly-only experimental API. (iter_map_windows)

Calls the given function f for each contiguous window of size N over self and returns an iterator over the outputs of f. Like slice::windows(), the windows during mapping overlap as well.

fn fuse(self) -> Fuse<Self>

where Self: Sized,

Creates an iterator which ends after the first None.

fn inspect<F>(self, f: F) -> Inspect<Self, F>

where Self: Sized, F: FnMut(&Self::Item),

Does something with each element of an iterator, passing the value on.

fn by_ref(&mut self) -> &mut Self

where Self: Sized,

Borrows an iterator, rather than consuming it.

fn collect<B>(self) -> B

where B: FromIterator<Self::Item>, Self: Sized,

Transforms an iterator into a collection.

fn collect_into<E>(self, collection: &mut E) -> &mut E

where E: Extend<Self::Item>, Self: Sized,

🔬This is a nightly-only experimental API. (iter_collect_into)

Collects all the items from an iterator into a collection.

fn partition<B, F>(self, f: F) -> (B, B)

where Self: Sized, B: Default + Extend<Self::Item>, F: FnMut(&Self::Item) -> bool,

Consumes an iterator, creating two collections from it.

fn is_partitioned<P>(self, predicate: P) -> bool

where Self: Sized, P: FnMut(Self::Item) -> bool,

🔬This is a nightly-only experimental API. (iter_is_partitioned)

Checks if the elements of this iterator are partitioned according to the given predicate, such that all those that return true precede all those that return false.

fn try_fold<B, F, R>(&mut self, init: B, f: F) -> R

where Self: Sized, F: FnMut(B, Self::Item) -> R, R: Try<Output = B>,

An iterator method that applies a function as long as it returns successfully, producing a single, final value.

fn try_for_each<F, R>(&mut self, f: F) -> R

where Self: Sized, F: FnMut(Self::Item) -> R, R: Try<Output = ()>,

An iterator method that applies a fallible function to each item in the iterator, stopping at the first error and returning that error.

fn fold<B, F>(self, init: B, f: F) -> B

where Self: Sized, F: FnMut(B, Self::Item) -> B,

Folds every element into an accumulator by applying an operation, returning the final result.

fn reduce<F>(self, f: F) -> Option<Self::Item>

where Self: Sized, F: FnMut(Self::Item, Self::Item) -> Self::Item,

Reduces the elements to a single one, by repeatedly applying a reducing operation.

fn try_reduce<R>( &mut self, f: impl FnMut(Self::Item, Self::Item) -> R, ) -> <<R as Try>::Residual as Residual<Option<<R as Try>::Output>>>::TryType

where Self: Sized, R: Try<Output = Self::Item>, <R as Try>::Residual: Residual<Option<Self::Item>>,

🔬This is a nightly-only experimental API. (iterator_try_reduce)

Reduces the elements to a single one by repeatedly applying a reducing operation. If the closure returns a failure, the failure is propagated back to the caller immediately.

fn all<F>(&mut self, f: F) -> bool

where Self: Sized, F: FnMut(Self::Item) -> bool,

Tests if every element of the iterator matches a predicate.

fn any<F>(&mut self, f: F) -> bool

where Self: Sized, F: FnMut(Self::Item) -> bool,

Tests if any element of the iterator matches a predicate.

fn find<P>(&mut self, predicate: P) -> Option<Self::Item>

where Self: Sized, P: FnMut(&Self::Item) -> bool,

Searches for an element of an iterator that satisfies a predicate.

fn find_map<B, F>(&mut self, f: F) -> Option<B>

where Self: Sized, F: FnMut(Self::Item) -> Option<B>,

Applies function to the elements of iterator and returns the first non-none result.

fn try_find<R>( &mut self, f: impl FnMut(&Self::Item) -> R, ) -> <<R as Try>::Residual as Residual<Option<Self::Item>>>::TryType

where Self: Sized, R: Try<Output = bool>, <R as Try>::Residual: Residual<Option<Self::Item>>,

🔬This is a nightly-only experimental API. (try_find)

Applies function to the elements of iterator and returns the first true result or the first error.

fn position<P>(&mut self, predicate: P) -> Option<usize>

where Self: Sized, P: FnMut(Self::Item) -> bool,

Searches for an element in an iterator, returning its index.

fn max_by_key<B, F>(self, f: F) -> Option<Self::Item>

where B: Ord, Self: Sized, F: FnMut(&Self::Item) -> B,

Returns the element that gives the maximum value from the specified function.

fn max_by<F>(self, compare: F) -> Option<Self::Item>

where Self: Sized, F: FnMut(&Self::Item, &Self::Item) -> Ordering,

Returns the element that gives the maximum value with respect to the specified comparison function.

fn min_by_key<B, F>(self, f: F) -> Option<Self::Item>

where B: Ord, Self: Sized, F: FnMut(&Self::Item) -> B,

Returns the element that gives the minimum value from the specified function.

fn min_by<F>(self, compare: F) -> Option<Self::Item>

where Self: Sized, F: FnMut(&Self::Item, &Self::Item) -> Ordering,

Returns the element that gives the minimum value with respect to the specified comparison function.

fn unzip<A, B, FromA, FromB>(self) -> (FromA, FromB)

where FromA: Default + Extend<A>, FromB: Default + Extend<B>, Self: Sized + Iterator<Item = (A, B)>,

Converts an iterator of pairs into a pair of containers.

fn copied<'a, T>(self) -> Copied<Self>

where T: 'a + Copy, Self: Sized + Iterator<Item = &'a T>,

Creates an iterator which copies all of its elements.

fn cloned<'a, T>(self) -> Cloned<Self>

where T: 'a + Clone, Self: Sized + Iterator<Item = &'a T>,

Creates an iterator which clones all of its elements.

fn array_chunks<const N: usize>(self) -> ArrayChunks<Self, N>

where Self: Sized,

🔬This is a nightly-only experimental API. (iter_array_chunks)

Returns an iterator over N elements of the iterator at a time.

fn sum<S>(self) -> S

where Self: Sized, S: Sum<Self::Item>,

Sums the elements of an iterator.

fn product<P>(self) -> P

where Self: Sized, P: Product<Self::Item>,

Iterates over the entire iterator, multiplying all the elements

fn cmp_by<I, F>(self, other: I, cmp: F) -> Ordering

where Self: Sized, I: IntoIterator, F: FnMut(Self::Item, <I as IntoIterator>::Item) -> Ordering,

🔬This is a nightly-only experimental API. (iter_order_by)

Lexicographically compares the elements of this Iterator with those of another with respect to the specified comparison function.

fn partial_cmp<I>(self, other: I) -> Option<Ordering>

where I: IntoIterator, Self::Item: PartialOrd<<I as IntoIterator>::Item>, Self: Sized,

Lexicographically compares the PartialOrd elements of this Iterator with those of another. The comparison works like short-circuit evaluation, returning a result without comparing the remaining elements. As soon as an order can be determined, the evaluation stops and a result is returned.

fn partial_cmp_by<I, F>(self, other: I, partial_cmp: F) -> Option<Ordering>

where Self: Sized, I: IntoIterator, F: FnMut(Self::Item, <I as IntoIterator>::Item) -> Option<Ordering>,

🔬This is a nightly-only experimental API. (iter_order_by)

Lexicographically compares the elements of this Iterator with those of another with respect to the specified comparison function.

fn eq<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialEq<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are equal to those of another.

fn eq_by<I, F>(self, other: I, eq: F) -> bool

where Self: Sized, I: IntoIterator, F: FnMut(Self::Item, <I as IntoIterator>::Item) -> bool,

🔬This is a nightly-only experimental API. (iter_order_by)

Determines if the elements of this Iterator are equal to those of another with respect to the specified equality function.

fn ne<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialEq<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are not equal to those of another.

fn lt<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialOrd<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are lexicographically less than those of another.

fn le<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialOrd<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are lexicographically less or equal to those of another.

fn gt<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialOrd<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are lexicographically greater than those of another.

fn ge<I>(self, other: I) -> bool

where I: IntoIterator, Self::Item: PartialOrd<<I as IntoIterator>::Item>, Self: Sized,

Determines if the elements of this Iterator are lexicographically greater than or equal to those of another.

fn is_sorted_by<F>(self, compare: F) -> bool

where Self: Sized, F: FnMut(&Self::Item, &Self::Item) -> bool,

Checks if the elements of this iterator are sorted using the given comparator function.

fn is_sorted_by_key<F, K>(self, f: F) -> bool

where Self: Sized, F: FnMut(Self::Item) -> K, K: PartialOrd,

Checks if the elements of this iterator are sorted using the given key extraction function.