Last week, I announced Tower Web, a web application framework for Rust that focuses on removing boilerplate. It uses Rust macros to generate necessary HTTP boilerplate so that the application can be written decoupled from HTTP concerns.

One of the stated goals was that it needed to work on stable Rust today.

This is what the API looked like when I announced it:

#[macro_use]
extern crate tower_web;

use tower_web::ServiceBuilder;

#[derive(Clone, Debug)]
struct HelloWorld;

impl_web! {
    impl HelloWorld {
        /// @get("/")
        fn hello_world(&self) -> Result<String, ()> {
            Ok("Hello world".to_string())
        }
    }
}

pub fn main() {
    let addr = "127.0.0.1:8080".parse().expect("Invalid address");
    println!("Listening on http://{}", addr);

    ServiceBuilder::new()
        .resource(HelloWorld)
        .run(&addr)
        .unwrap();
}

Note that strange doc comment @get("/"). This has semantic meaning and that comment is how Tower Web matches HTTP requests with handler methods. It also isn’t very idiomatic Rust.

The short version is, Tower Web 0.2 was just released and regular Rust attributes are now used instead of magic comments. The doc comment is replaced with #[get("/")]. This is thanks to Rust macro wizard David Tolnay. I also thought that it would be best to immediately push out 0.2 and then we can all pretend 0.1 didn’t happen.

An incredible journey

But how did we get here? Let’s look back…

Historically, Rust macros have been very useful but limited in what they can enable. A macro like Tower Web’s impl_web!macro is definitely out of scope. Not that long ago, Rust introduced a limited form of procedural macros enabling libraries to provide custom derives. This was stabilized so that serde and diesel (among others) could target stable Rust. The scope was limited to defining a macro to handle #[derive(MyTrait)].

Of course, it didn’t take long to figure how to use custom derives to allow writing all sorts of macros that work with stable rust (and it is definitely a pretty intense and ingenious hack).

Tower Web — Take 1

When I started out working on Tower Web, I opted for real attributes. However, stable Rust does not permit attributes not known to the compiler. Because of this, the macro needs to strip all attributes. Stripping attributes with a proc macro is pretty easy thanks to syn. This is what it looks like:

fn clean(input: TokenStream) -> TokenStream {
    use syn;

    let ast = syn::parse2(input).unwrap();

    struct Clean;

    impl syn::fold::Fold for Clean {
        fn fold_impl_item_method(&mut self, mut item: syn::ImplItemMethod) -> syn::ImplItemMethod {
            item.attrs.clear();
            item
        }
    }

    let file = syn::fold::fold_file(&mut Clean, ast);
    quote! { #file }
}

That function removes all attributes from methods using syn’s fold helper. In practice, the function should only strip Tower Web’s attributes (so that attributes like #[inline] are maintained). Also, attributes would need to be stripped from more locations than just methods.

This stripping strategy works, but comes with a significant downside. Given the following method which includes an error in the definition,

#[get("/motd")]
fn motd(&self) -> Result<String, ()> {
    fn format(s: &str) -> String {
        format!("MOTD: {}", self.motd)
        s.to_string()
    }

    let motd = s(1);

    // You can also respond with an owned `String`.
    Ok(format!("MOTD: {}", motd))
}

compiling results in the following output.

$cargo check --example hello_world
    Checking tower-web v0.1.2 (file:///Users/carllerche/Code/Oss/Tokio/tower-web)
error: proc-macro derive panicked
   --> examples/hello_world.rs:71:1
    |
71  | / impl_web! {
72  | |     impl HelloWorld {
73  | |
74  | |         // `hello_world` is a plain old method on `HelloWorld`. However, note
...   |
125 | |     }
126 | | }
    | |_^
    |
    = help: message: called `Result::unwrap()` on an `Err` value: ParseError(Some("failed to parse crate: failed to parse anything"))
    = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to previous error

error: Could not compile `tower-web`.

To learn more, run the command again with --verbose.

In other words, using a proc macro to strip the attributes from the method results in losing useful compiler error messages. This makes the approach a non starter.

That is why I opted to use the comment approach for attributes. When using comments to annotate handler methods, the contents of impl_web! did not need to be touched by the macro. This avoided having to pass it through the proc macro for processing. In Tower Web 0.1, the proc macro only reads the user code and generates the resource implementation based on that. The macro is here.

This worked, but resulted in an unidiomatic Rust API.

Tower Web — Take 2

There was some discussion on Reddit about the doc comment approach. Eventually, a PR was submitted that provided a regular Rust macro that could strip the attributes. Because it is a regular Rust macro, error messages are preserved. The actual macro is not trivial, but it works.

There is one downside. Regular Rust macros have a recursion limit. In complex cases, the macro to strip attributes can hit this limit and compilation will fail. The user can work around this by adding #![recursion_limit="128"] to their code. This is just a temporary issue. Macros 1.2 are on the path to stabilization and with them will come true attribute proc macros. Once this lands on stable, Tower Web will switch to using that and all will be good.

Since it has been less than a week since Tower Web 0.1 was released, I thought it would be best to just release this change as 0.2 and pretend 0.1 never happened. Hopefully nobody wrote and shipped an app to production already (though I would be impressed).