Struct syntax::tokenstream::TokenStream

pub struct TokenStream {
    kind: TokenStreamKind,
}

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Token Streams

A TokenStream is an abstract sequence of tokens, organized into TokenTrees. The goal is for procedural macros to work with TokenStreams and TokenTrees instead of a representation of the abstract syntax tree. Today's TokenTrees can still contain AST via Token::Interpolated for back-compat.

Fields

kind: TokenStreamKind

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Methods

impl TokenStream

pub fn len(&self) -> usize

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn empty() -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn is_empty(&self) -> bool

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn concat(streams: Vec<TokenStream>) -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

fn concat_rc_slice(streams: RcSlice<TokenStream>) -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn trees(&self) -> Cursor

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn into_trees(self) -> Cursor

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn eq_unspanned(&self, other: &TokenStream) -> bool

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Compares two TokenStreams, checking equality without regarding span information.

pub fn probably_equal_for_proc_macro(&self, other: &TokenStream) -> bool

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn as_tree(self) -> (TokenTree, bool)

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Precondition: self consists of a single token tree. Returns true if the token tree is a joint operation w.r.t. proc_macro::TokenNode.

pub fn map_enumerated<F: FnMut(usize, TokenTree) -> TokenTree>(     self,     f: F ) -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

pub fn map<F: FnMut(TokenTree) -> TokenTree>(self, f: F) -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

fn first_tree_and_joint(&self) -> Option<(TokenTree, bool)>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

fn last_tree_if_joint(&self) -> Option<TokenTree>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl From<TokenTree> for TokenStream

fn from(tt: TokenTree) -> TokenStream

Performs the conversion.

impl From<Token> for TokenStream

fn from(token: Token) -> TokenStream

Performs the conversion.

impl<T: Into<TokenStream>> FromIterator<T> for TokenStream

fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self

Creates a value from an iterator.

impl Eq for TokenStream

fn assert_receiver_is_total_eq(&self)

impl PartialEq<TokenStream> for TokenStream

fn eq(&self, other: &TokenStream) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=.

impl From<TokenStream> for ThinTokenStream

fn from(stream: TokenStream) -> ThinTokenStream

Performs the conversion.

impl From<ThinTokenStream> for TokenStream

fn from(stream: ThinTokenStream) -> TokenStream

Performs the conversion.

impl Display for TokenStream

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl Encodable for TokenStream

fn encode<E: Encoder>(&self, encoder: &mut E) -> Result<(), E::Error>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

impl Decodable for TokenStream

fn decode<D: Decoder>(decoder: &mut D) -> Result<TokenStream, D::Error>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

impl Hash for TokenStream

fn hash<H: Hasher>(&self, state: &mut H)

Feeds this value into the given [Hasher].

fn hash_slice<H>(data: &[Self], state: &mut H) where     H: Hasher,

Feeds a slice of this type into the given [Hasher].