Crate icu_provider

icu_provider is one of the ICU4X components.

Unicode’s experience with ICU4X’s parent projects, ICU4C and ICU4J, led the team to realize that data management is the most critical aspect of deploying internationalization, and that it requires a high level of customization for the needs of the platform it is embedded in. As a result ICU4X comes with a selection of providers that should allow for ICU4X to naturally fit into different business and technological needs of customers.

icu_provider defines traits and structs for transmitting data through the ICU4X locale data pipeline. The primary trait is DataProvider. It is parameterized by a DataMarker, which is the type-system-level data identifier. DataProvider has a single method, DataProvider::load, which transforms a DataRequest into a DataResponse.

• DataRequest contains selectors to choose a specific variant of the marker, such as a locale.
• DataResponse contains the data if the request was successful.

The most common types required for this crate are included via the prelude:

use icu_provider::prelude::*;

Dynamic Data Providers

If the type system cannot be leveraged to load data (such as when dynamically loading from I/O), there’s another form of the DataProvider: DynamicDataProvider. While DataProvider is parametrized on the type-system level by a DataMarker (which are distinct types implementing this trait), DynamicDataProviders are parametrized at runtime by a DataMarkerInfo struct, which essentially is the runtime representation of the DataMarker type.

The DynamicDataProvider is still type-level parametrized by the type that it loads, and there are two implementations that should be called out

• DynamicDataProvider<BufferMarker>, a.k.a. BufferProvider returns data as [u8] buffers.

BufferProvider

These providers are able to return unstructured data typically represented as serde-serialized buffers. Users can call as_deserializing() to get an object implementing DataProvider by invoking Serde Deserialize.

Examples of BufferProviders:

• FsDataProvider reads individual buffers from the filesystem.
• BlobDataProvider reads buffers from a large in-memory blob.

Provider Adapters

ICU4X offers several built-in modules to combine providers in interesting ways. These can be found in the icu_provider_adapters crate.

Testing Provider

This crate also contains a concrete provider for demonstration purposes:

• HelloWorldProvider returns “hello world” strings in several languages.

Types and Lifetimes

Types compatible with Yokeable can be passed through the data provider, so long as they are associated with a marker type implementing DynamicDataMarker.

Data structs should generally have one lifetime argument: 'data. This lifetime allows data structs to borrow zero-copy data.

Modules

baked

This module contains scaffolding for baked providers, typically generated using databake.

buf

Traits for data providers that produce opaque buffers.

constructors

📚 This module documents ICU4X constructor signatures.

dynutil

Utilities for using trait objects with DataPayload.

export

This module contains types required to export ICU4X data via the icu_provider_export crate. End users should not need to consume anything in this module.

hello_world

Data provider returning multilingual “Hello World” strings for testing.

marker

Additional DataMarker helpers.

prelude

Core selection of APIs and structures for the ICU4X data provider.

ule

Traits that data provider implementations can use to optimize storage by using VarULE.

Macros

data_marker

Creates a data marker.

data_struct

Implements required traits on data structs, such as MaybeEncodeAsVarULE.

Structs

AttributeParseError

Invalid character

Cart

The type of the “cart” that is used by DataPayload.

DataError

The error type for ICU4X data provider operations.

DataIdentifierBorrowed

The borrowed version of a DataIdentifierCow.

DataIdentifierCow

A data identifier identifies a particular version of data, such as “English”.

DataLocale

A locale type optimized for use in fallbacking and the ICU4X data pipeline.

DataMarkerAttributes

An additional key to identify data beyond a DataLocale.

DataMarkerInfo

Used for loading data from a dynamic ICU4X data provider.

DataPayload

A container for data payloads returned from a data provider.

DataProviderWithMarker

A DataProvider associated with a specific marker.

DataRequest

The request type passed into all data provider implementations.

DataRequestMetadata

Metadata for data requests. This is currently empty, but it may be extended with options for tuning locale fallback, buffer layout, and so forth.

DataResponse

A response object containing an object as payload and metadata about it.

DataResponseMetadata

A response object containing metadata about the returned data.

Enums

DataErrorKind

A list specifying general categories of data provider error.

Traits

BoundDataProvider

A data provider that loads data for a specific data type.

DataMarker

A DynamicDataMarker with a DataMarkerInfo attached.

DataProvider

A data provider that loads data for a specific DataMarkerInfo.

DryDataProvider

A data provider that can determine whether it can load a particular data identifier, potentially cheaper than actually performing the load.

DynamicDataMarker

Trait marker for data structs. All types delivered by the data provider must be associated with something implementing this trait.

DynamicDataProvider

A data provider that loads data for a specific data type.

DynamicDryDataProvider

A dynanmic data provider that can determine whether it can load a particular data identifier, potentially cheaper than actually performing the load.

IterableDataProvider

A DataProvider that can iterate over all supported DataIdentifierCows.

IterableDynamicDataProvider

A DynamicDataProvider that can iterate over all supported DataIdentifierCows for a certain marker.

ResultDataError

Extension trait for Result<T, DataError>.