kochab/src/lib.rs

1076 lines
42 KiB
Rust
Raw Normal View History

#![warn(missing_docs)]
#![doc(html_logo_url = "https://raw.githubusercontent.com/twitter/twemoji/master/assets/svg/1f30c.svg")]
//! Kochab is an ergonomic and intuitive library for quickly building highly functional
//! and advanced Gemini applications on either SCGI or raw Gemini.
//!
//! Originally based on [northstar](https://crates.io/crates/northstar), though now far
2020-12-05 15:30:13 +00:00
//! diverged, kochab offers many features to help you build your application without any
//! of the boilerplate or counterintuitive shenanigans. Kochab centers around it's many
//! feature flags, which let you pick out exactly the features you need to build your
//! library, while leaving out features you don't need to keep your application
//! bloat-free.
//!
//! Another central feature of kochab is its multi-protocol abstraction. An application
//! built using kochab can easily compile either as a gemini application or as a SCGI
//! script using only a single feature flag.
//!
//! ## Features
//!
//! Kochab offers a wide array of features, so don't get overwhelmed. By default, you
//! start off with only the `gemini_srv` feature, and you're able to add on more features
//! as you need them. All of kochab's features are documented below.
//!
//! * `ratelimiting` - The ratelimiting feature adds in the ability to limit how often
//! users can access certain areas of an application. This is primarily configured using
//! the [`Server::ratelimit()`] method.
//!
2020-12-05 15:30:13 +00:00
//! * `serve_dir` - Adds in utilities for serving files & directories from the disk at
//! runtime. The easiest way to use this is to pass a [`PathBuf`] to the
//! [`Server::add_route()`] method, which will either serve a directory or a single file.
//! Files and directories can also be served using the methods in the [`util`] module.
//!
//! * `user_management` - Adds in tools to manage users using a certificate authentication
//! system. The user management suite is one of kocab's biggest features. When active,
//! kochab will maintain a database of registered users, linking each to a certificate.
//! Users also have custom data associated with them, which can be retrieved and modified
//! by the application.
//!
//! * `user_management_advanced` - Allows users to set a password and add additional
//! certificates. Without this feature, one certificate can only have one linked account,
//! unless you manually implement an authentication system. With this feature, kochab
//! will use argon2 to hash and check user passwords, and store user passwords alongside
//! the other user data
//!
//! * `user_management_routes` - The user management routes feature automates much of the
//! hard work of connecting the tools provided with the other two features with endpoints
//! that the user connects to. Kochab will manage all requests to the `/account` route,
//! and create pages to allow users to create an account, link new certificates, or
//! change/set their password. This also adds the ability to set an authenticated route,
//! which will automatically prompt the user to sign in, and give your handler access to
//! the user's data with no added work. This can be used with either the
//! `user_management_advanced` feature, or just the basic `user_management` feature
//!
//! * `certgen` - Enables automatically generating TLS certificates. Since only servers
//! directly using the gemini protocol need TLS certificates, this implies `gemini_srv`,
//! and should not be used with `scgi_srv`. By default, kochab will try to generate a
//! certificate by prompting the user in stdin/stdout, but this behavior can be customized
//! using [`Server::set_certificate_generation_mode()`].
//!
//! * `dashmap` - Enables some minor optimizations within the `user_management_routes`
//! feature. Automatically enabled by `ratelimiting`.
//!
//! * `ring` - When using `user_management_advanced` with `scgi_srv`, salts are calculated
//! based off of a simple PRNG and the system time. This should be plenty secure enough,
//! especially since we're using a good number argon2 rounds, but for bonus paranoia
//! points, you can add ring as a dependency to source secure random. This is enabled
//! automatically on `gemini_srv`, since `ring` is added as a dependency for certificate
//! processing. When not using the `user_management_advanced` feature, this does nothing
//! but increase your build time & size.
//!
//! * `gemini_srv`/`scgi_srv` - Switches between serving content using SCGI and serving
//! content as a raw gemini server. One and only one of these features must be enabled,
//! and compilation will fail if both are enabled. See below for more information.
2020-12-05 15:30:13 +00:00
//!
//! ## Gemini & SCGI Modes
//!
//! **It is highly recommended that you read this section, *especially* if you don't know
//! what SCGI is.**
//!
//! Central to kochab's repertoire is the ability to serve either content either through
//! raw Gemini or through a reverse proxy with SCGI. This can be accomplished easily
//! using a single feature flag. But first:
//!
//! ### What's SCGI and why should I be using it?
//!
//! You're probably familiar with the Gemini protocol, and how servers serve things on it.
//! Gemini is easy to serve and simple to work with. You probably went into this project
//! expecting to serve content directly through the Gemini protocol. So what's this about
//! SCGI, and why should you bother using it?
//!
//! The problem that SCGI solves is that it is very difficult to have multiple servers on
//! one domain with Gemini. For example, if you wanted to serve your blog on `/blog.gmi`,
//! but wanted to have an app running on `/app`, you would most likely have to use SCGI.
//!
//! SCGI has two parts: A main gemini server (called the SCGI client) that handles TLS
//! for incoming connections and either serves some static content, runs some other
//! handler, or passes it off to the SCGI server, which is what you'd be writing. The
//! SCGI server doesn't directly interact with the client, but gets plenty of information
//! about the connection from the server, like what path the server is being served on
//! (like "/app"), the end user's IP, and the certificate fingerprint being used, if
//! applicable.
//!
//! Because SCGI servers don't need to bother with TLS, compiling your app as SCGI will
//! also be fairly faster, since kochab doesn't need to bring in `rustls` or any
//! TLS certificate generation libraries.
//!
//! This doesn't necessarily mean that SCGI is for every app, but if you suspect that a
//! user might ever need to run your app on a path other than the main one, or may want to
//! serve static files alongside your site, I encourage you to seriously consider it.
//!
//! ### But I don't want to bother learning a new protocol ![blobcat-pout]
//!
//! You don't have to! Kochab handles everything about it, thanks to the magic of
//! abstraction! The only thing you have to change are the feature flags in your
//! `Config.toml`. In fact, you could even expose the feature flag so that users can
//! compile your crate as either a Gemini or SCGI server without needing to write any
//! conditional compilation!
//!
//! ### Getting started
//!
//! **Updating your feature flags**
//!
//! By default, Kochab serves content over raw gemini, to make it easier for new users to
//! jump right into using the library. This is done using the on-by-default `gemini_srv`
//! feature flag. To switch to SCGI, we want to switch off `gemini_srv` and switch on
//! `scgi_srv`.
//!
//! ```toml
//! [dependencies.kochab]
//! git = "https://gitlab.com/Alch_Emi/kochab.git"
//! branch = "stable"
//! default-features = false
//! features = ["scgi_srv"] # and any other features you might need
//! ```
//!
//! **Testing with a minimal SCGI client**
//!
//! To give your code a run, you'll need a server to handle Gemini requests and pass them
//! off to your SCGI server. There's a few Gemini servers out there with SCGI support,
//! but if you're just interested in giving your code a quick run, I'd recommend
//! stargazer, which has very good SCGI support and is super easy to set up if you are
//! already using cargo.
2020-12-05 15:30:13 +00:00
//!
//! You can install stargazer by running.
//! ```sh
//! cargo install stargazer
//! ```
2020-12-05 15:30:13 +00:00
//!
//! Once you have it, you can find a super simple configuration file [here][2], and then
//! just run
//!
//! ```sh
//! stargazer -C stargazer.ini
2020-12-05 15:30:13 +00:00
//! ```
//!
//! Now, when you run your code, you can connect to `localhost`, and molly brown will
//! connect to your SCGI server and forward the response on to your Gemini client.
//!
//! **Rewriting Paths**
//!
//! One important difference about writing code for an SCGI server is that your app might
//! be served on a path that's not the base path. If this happens, you suddenly have a
//! distinction between an absolute link for your app, and an absolute link for the gemini
//! server.
//!
//! For example, if an app that's being served on `/app` has a link to `/path`, this could
//! be either:
//!
//! * Meant to be handled by the app by the route at `/path`, which would be `/app/path`
//! for the parent Gemini server, or
//! * Meant to link to some content on the parent gemini server, at `/path`, which would
//! mean linking to a url your app doesn't control
//!
//! Most of the time, you want to do the first one. Thankfully, Kochab makes rewriting
//! links relative to the base of your app super easy. In most cases, all you need is to
//! add a single line to your Server builder pattern.
//!
//! For more information, see [`Server::set_autorewrite()`].
//!
//! [1]: https://tildegit.org/solderpunk/molly-brown
//! [2]: https://gitlab.com/Alch_Emi/kochab/-/raw/devel/stargazer.ini
2020-12-05 15:30:13 +00:00
//! [blobcat-pout]: https://the-apothecary.club/_matrix/media/r0/thumbnail/the-apothecary.club/10a406405a5bcd699a5328259133bfd9260320a6?height=99&width=20 ":blobcat-pout:"
//! <style>
//! img[alt=blobcat-pout] { width: 20px; vertical-align: top; }
//! </style>
2020-10-31 19:53:03 +00:00
#[macro_use] extern crate log;
#[cfg(all(feature = "gemini_srv", feature = "scgi_srv", not(doc)))]
compile_error!("Please enable only one of either the `gemini_srv` or `scgi_srv` features on the kochab crate");
#[cfg(not(any(feature = "gemini_srv", feature = "scgi_srv")))]
compile_error!("Please enable at least one of either the `gemini_srv` or `scgi_srv` features on the kochab crate");
use std::{
time::Duration,
2020-12-03 20:04:12 +00:00
future::Future,
};
#[cfg(feature = "gemini_srv")]
use std::convert::TryFrom;
#[cfg(feature = "scgi_srv")]
use std::{
collections::HashMap,
net::SocketAddr,
str::FromStr,
};
#[cfg(any(feature = "gemini_srv", feature = "user_management"))]
use std::path::PathBuf;
#[cfg(feature="ratelimiting")]
use std::net::IpAddr;
2020-10-31 19:53:03 +00:00
use tokio::{
io,
io::BufReader,
net::TcpListener,
net::ToSocketAddrs,
2020-12-03 20:04:12 +00:00
time,
2020-10-31 19:53:03 +00:00
prelude::*,
};
#[cfg(feature = "scgi_srv")]
use tokio::net::UnixListener;
#[cfg(feature = "gemini_srv")]
use tokio::{
time::timeout,
net::TcpStream,
2020-10-31 19:53:03 +00:00
};
#[cfg(feature = "ratelimiting")]
use tokio::time::interval;
2020-10-31 19:53:03 +00:00
use anyhow::*;
use routing::RoutingNode;
#[cfg(feature = "ratelimiting")]
use ratelimiting::RateLimiter;
#[cfg(feature = "gemini_srv")]
use tokio_rustls::TlsAcceptor;
#[cfg(feature = "gemini_srv")]
use rustls::Session;
2020-10-31 19:53:03 +00:00
mod types;
2020-12-06 15:24:54 +00:00
mod handling;
2020-10-31 19:53:03 +00:00
pub mod util;
2020-11-20 02:37:02 +00:00
pub mod routing;
#[cfg(feature = "ratelimiting")]
2020-12-06 15:24:54 +00:00
mod ratelimiting;
2020-11-16 06:13:16 +00:00
#[cfg(feature = "user_management")]
pub mod user_management;
#[cfg(feature = "gemini_srv")]
mod cert;
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
use user_management::UserManager;
#[cfg(feature = "certgen")]
pub use cert::CertGenMode;
2020-10-31 19:53:03 +00:00
2020-12-07 20:58:49 +00:00
pub use uriparse::URIReference;
2020-10-31 19:53:03 +00:00
pub use types::*;
2020-12-06 15:24:54 +00:00
pub use handling::Handler;
2020-10-31 19:53:03 +00:00
/// The maximun length of a Request URI
2020-10-31 19:53:03 +00:00
pub const REQUEST_URI_MAX_LEN: usize = 1024;
/// The default port for the gemini protocol
2020-10-31 19:53:03 +00:00
pub const GEMINI_PORT: u16 = 1965;
struct ServerInner {
#[cfg(feature = "gemini_srv")]
2020-10-31 19:53:03 +00:00
tls_acceptor: TlsAcceptor,
routes: RoutingNode<Handler>,
timeout: Duration,
complex_timeout: Option<Duration>,
2020-12-15 15:32:31 +00:00
#[cfg(feature = "scgi_srv")]
autorewrite: bool,
#[cfg(feature="ratelimiting")]
rate_limits: RoutingNode<RateLimiter<IpAddr>>,
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
manager: UserManager,
2020-10-31 19:53:03 +00:00
}
impl ServerInner {
async fn serve_ip(self, listener: TcpListener) -> Result<()> {
let static_self: &'static Self = Box::leak(Box::new(self));
#[cfg(feature = "ratelimiting")]
tokio::spawn(prune_ratelimit_log(&static_self.rate_limits));
2020-10-31 19:53:03 +00:00
loop {
let (stream, _addr) = listener.accept().await
2020-11-14 02:56:50 +00:00
.context("Failed to accept client")?;
2020-10-31 19:53:03 +00:00
tokio::spawn(async move {
if let Err(err) = static_self.serve_client(stream).await {
2020-11-14 02:56:50 +00:00
error!("{:?}", err);
2020-10-31 19:53:03 +00:00
}
});
}
}
#[cfg(feature = "scgi_srv")]
// Yeah it's code duplication, but I can't find a way around it, so this is what we're
// getting for now
async fn serve_unix(self, listener: UnixListener) -> Result<()> {
let static_self: &'static Self = Box::leak(Box::new(self));
#[cfg(feature = "ratelimiting")]
tokio::spawn(prune_ratelimit_log(&static_self.rate_limits));
loop {
let (stream, _addr) = listener.accept().await
.context("Failed to accept client")?;
tokio::spawn(async move {
if let Err(err) = static_self.serve_client(stream).await {
error!("{:?}", err);
}
});
}
}
async fn serve_client(
&'static self,
#[cfg(feature = "gemini_srv")]
stream: TcpStream,
#[cfg(all(feature = "scgi_srv", not(feature = "gemini_srv")))]
stream: impl AsyncWrite + AsyncRead + Unpin + Send,
) -> Result<()> {
let fut_accept_request = async {
#[cfg(feature = "gemini_srv")]
let stream = self.tls_acceptor.accept(stream).await
.context("Failed to establish TLS session")?;
let mut stream = BufReader::new(stream);
let request = self.receive_request(&mut stream).await
.context("Failed to receive request")?;
Result::<_, anyhow::Error>::Ok((request, stream))
};
2020-10-31 19:53:03 +00:00
// Wait for the request to be parsed
let (mut request, mut stream) = {
#[cfg(feature = "gemini_srv")] {
// Use a timeout for interacting with the client
let fut_accept_request = timeout(self.timeout, fut_accept_request);
fut_accept_request.await
.context("Client timed out while waiting for response")??
}
#[cfg(feature = "scgi_srv")]
fut_accept_request.await?
};
// Determine the remote client's IP address for logging and ratelimiting
let peer_addr = {
#[cfg(feature = "gemini_srv")] {
stream.get_ref()
.get_ref()
.0
.peer_addr()?
.ip()
}
#[cfg(feature = "scgi_srv")] {
let remote = request.headers()
.get("REMOTE_ADDR")
.ok_or(ParseError::Malformed("REMOTE_ADDR header not received"))?
.as_str();
SocketAddr::from_str(remote)
.map(|a| a.ip())
.or_else(|_| std::net::IpAddr::from_str(remote))
.context("Received malformed IP address from upstream")?
}
};
#[cfg(feature = "ratelimiting")]
// Perform ratelimiting checks
if let Some(resp) = self.check_rate_limits(peer_addr, &request) {
// Log warning
warn!(
"Client from {} requesting {} was turned away by ratelimiting",
peer_addr,
request.uri()
);
// Send error response
self.send_response(resp, &mut stream).await
.context("Failed to send response")?;
// Exit
return Ok(())
}
info!("{} requested: {}", peer_addr, request.uri());
2020-10-31 19:53:03 +00:00
2020-11-13 19:20:59 +00:00
// Identify the client certificate from the tls stream. This is the first
// certificate in the certificate chain.
#[cfg(feature = "gemini_srv")] { // This is done earlier for `scgi_srv`
let client_cert = stream.get_ref()
.get_ref()
.1
.get_peer_certificates()
.and_then(|mut v| if v.is_empty() {None} else {Some(v.remove(0))});
request.set_cert(client_cert);
}
#[cfg_attr(feature = "gemini_srv", allow(unused_mut))] // Used for scgi_srv only
let mut response = if let Some((trailing, handler)) = self.routes.match_request(&request) {
2020-11-20 18:22:34 +00:00
request.set_trailing(trailing);
handler.handle(request.clone()).await
} else {
Response::not_found()
};
#[cfg(feature = "scgi_srv")] // No point running noop code
if self.autorewrite {
match response.rewrite_all(&request).await {
Ok(true) => { /* all is well */ }
Ok(false) => {
error!(
concat!(
"Upstream did not include SCRIPT_PATH or SCRIPT_NAME, refusing to",
" serve any text/gemini content with absolute links. It's most",
" likely that the proxy server you're using doesn't correctly",
" or completely implement the SCGI specification.",
)
);
response = Response::temporary_failure("Server misconfigured");
},
Err(e) => {
error!("Error reading text/gemini file from Response reader: {}", e);
response = Response::not_found();
}
}
}
self.send_response(response, &mut stream).await
.context("Failed to send response")?;
Ok(())
}
2020-12-15 14:13:11 +00:00
#[allow(clippy::useless_let_if_seq)]
async fn send_response(&self, response: Response, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
2020-11-19 18:29:17 +00:00
let use_complex_timeout =
response.body.is_some() &&
response.meta != "text/plain" &&
response.meta != "text/gemini" &&
2020-11-19 18:29:17 +00:00
self.complex_timeout.is_some();
2020-11-19 18:29:17 +00:00
let send_general_timeout;
let send_header_timeout;
let send_body_timeout;
2020-11-19 18:29:17 +00:00
if use_complex_timeout {
send_general_timeout = None;
send_header_timeout = Some(self.timeout);
send_body_timeout = self.complex_timeout;
} else {
send_general_timeout = Some(self.timeout);
send_header_timeout = None;
send_body_timeout = None;
}
if !response.is_success() && response.body.is_some() {
warn!(concat!(
"Received a response with a body, but a status code of {} (!= 20). ",
" Responses should only have a body if their status code is 20. The body",
" will be sent, but this is likely to cause unexpected behavior."),
response.status,
);
}
2020-11-19 18:29:17 +00:00
opt_timeout(send_general_timeout, async {
// Send the header
opt_timeout(send_header_timeout, send_response_header(&response, stream))
2020-11-19 18:29:17 +00:00
.await
.context("Timed out while sending response header")?
.context("Failed to write response header")?;
2020-11-19 18:29:17 +00:00
// Send the body
opt_timeout(send_body_timeout, send_response_body(response.body, stream))
.await
2020-11-19 18:29:17 +00:00
.context("Timed out while sending response body")?
.context("Failed to write response body")?;
2020-10-31 19:53:03 +00:00
2020-11-19 18:29:17 +00:00
Ok::<_,Error>(())
})
.await
.context("Timed out while sending response data")??;
2020-11-19 18:29:17 +00:00
Ok(())
2020-10-31 19:53:03 +00:00
}
#[cfg(feature="ratelimiting")]
fn check_rate_limits(&self, addr: IpAddr, req: &Request) -> Option<Response> {
if let Some((_, limiter)) = self.rate_limits.match_request(req) {
if let Err(when) = limiter.check_key(addr) {
return Some(Response::slow_down(when.as_secs()))
}
}
None
}
#[cfg(feature = "gemini_srv")]
2020-11-16 06:13:16 +00:00
async fn receive_request(
&'static self,
stream: &mut (impl AsyncBufRead + Unpin + Send),
2020-11-16 06:13:16 +00:00
) -> Result<Request> {
const HEADER_LIMIT: usize = REQUEST_URI_MAX_LEN + "\r\n".len();
let mut stream = stream.take(HEADER_LIMIT as u64);
2020-11-16 06:13:16 +00:00
let mut uri = Vec::new();
stream.read_until(b'\n', &mut uri).await?;
if !uri.ends_with(b"\r\n") {
if uri.len() < REQUEST_URI_MAX_LEN {
bail!("Request header not terminated with CRLF")
} else {
bail!("Request URI too long")
}
}
// Strip CRLF
uri.pop();
uri.pop();
let uri = URIReference::try_from(&*uri)
.context("Request URI is invalid")?
.into_owned();
Request::new(
2020-11-16 06:13:16 +00:00
uri,
#[cfg(feature="user_management")]
&self.manager,
).context("Failed to create request from URI")
}
#[cfg(feature = "scgi_srv")]
async fn receive_request(
&'static self,
stream: &mut (impl AsyncBufRead + Unpin),
) -> Result<Request> {
let mut buff = Vec::with_capacity(4);
#[allow(clippy::char_lit_as_u8)]
// Read the length of the header netstring (e.g. "120:")
stream.read_until(':' as u8, &mut buff).await?;
buff.pop(); // Remove the trailing ':'
let len = std::str::from_utf8(&*buff)
.ok()
.and_then(|s| usize::from_str(s).ok())
.ok_or(ParseError::Malformed("netstring length"))?;
// Read in the headers
buff.clear();
buff.resize(len + 1, 0);
stream.read_exact(buff.as_mut()).await?;
buff.truncate(len - 1); // Remove the final \x00,
// Parse the headers
let (maybe_trailing, headers) = buff.split(|b| *b == 0) // Headers are null delimiited
.map(|bytes| // Convert to an &str
std::str::from_utf8(bytes)
.map_err(|_| ParseError::Malformed("scgi headers"))
.map(str::trim)
)
.try_fold( // Turn the array of [header, value, header, ...] into a map
(Option::<&str>::None, HashMap::<String, String>::with_capacity(16)),
|(last_header, mut headers), s| {
s.map(|text| {
match last_header {
None => (Some(text), headers),
Some(header) => {
headers.insert(header.to_string(), text.to_string());
(None, headers)
}
}
})
}
)?;
// If there's not the same number of headers as values, that's a problem
if maybe_trailing.is_some() {
bail!(ParseError::Malformed("trailing header"));
}
// Check the content length info
let cont_len_val = headers.get("CONTENT_LENGTH")
.ok_or(ParseError::Malformed("No content length header!"))?;
let cont_len = usize::from_str(cont_len_val)
.map_err(|_| ParseError::Malformed("Malformed content length"))?;
if cont_len > 0 {
bail!(ParseError::Malformed("Gemini SCGI requests should not have a body"));
}
// Spec requires setting an SCGI header to one
if *headers.get("SCGI").ok_or(ParseError::Malformed("No SCGI header"))? != "1" {
bail!(ParseError::Malformed("SCGI header not set to \"1\""));
}
2020-11-16 06:13:16 +00:00
trace!("Headers received: {:?}", headers);
Ok(
Request::new(
headers,
#[cfg(feature = "user_management")]
&self.manager,
)?
)
2020-11-16 06:13:16 +00:00
}
2020-10-31 19:53:03 +00:00
}
#[derive(Debug)]
#[cfg(feature = "scgi_srv")]
enum ParseError {
IO(io::Error),
Malformed(&'static str),
}
#[cfg(feature = "scgi_srv")]
impl From<io::Error> for ParseError {
fn from(e: io::Error) -> Self {
Self::IO(e)
}
}
#[cfg(feature = "scgi_srv")]
impl std::fmt::Display for ParseError {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
match self {
Self::IO(e) => write!(f, "IO Error while parsing and responding SCGI: {}", e),
Self::Malformed(e) => write!(f, "SCGI request malformed at {}", e),
}
}
}
#[cfg(feature = "scgi_srv")]
impl std::error::Error for ParseError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
if let Self::IO(e) = self { Some(e) } else { None }
}
}
2020-12-07 21:46:13 +00:00
/// A builder for configuring a kochab server
///
/// Once created with [`Server::new()`], different configuration methods can be
/// called to set up the server, before finally making a call to [`serve_ip()`] or
/// [`serve_unix()`].
///
/// Technically, no methods need to be called in order to create the server, but unless
/// you add at least one route with [`add_route()`], the server will respond with `51 NOT
/// FOUND` to all requests.
///
/// # Example
/// ```no_run
/// use anyhow::Result;
/// # use kochab::Response;
/// # use kochab::Request;
/// # use kochab::Server;
///
/// #[tokio::main]
/// async fn main() {
/// Server::new()
/// .add_route("/", hello_world)
/// .serve_ip("localhost:1965")
/// .await;
/// }
///
/// async fn hello_world(_: Request) -> Result<Response> {
/// Ok(Response::success_gemini("Hello world!"))
/// }
/// ```
///
/// [`serve_ip()`]: Self::serve_ip()
/// [`serve_unix()`]: Self::serve_unix()
/// [`add_route()`]: Self::add_route()
pub struct Server {
timeout: Duration,
complex_body_timeout_override: Option<Duration>,
2020-11-20 18:54:24 +00:00
routes: RoutingNode<Handler>,
2020-12-15 15:32:31 +00:00
#[cfg(feature = "scgi_srv")]
autorewrite: bool,
#[cfg(feature = "gemini_srv")]
cert_path: PathBuf,
#[cfg(feature = "gemini_srv")]
key_path: PathBuf,
#[cfg(feature="ratelimiting")]
rate_limits: RoutingNode<RateLimiter<IpAddr>>,
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
data_dir: PathBuf,
#[cfg(feature="user_management")]
database: Option<sled::Db>,
#[cfg(feature="certgen")]
certgen_mode: CertGenMode,
2020-10-31 19:53:03 +00:00
}
impl Server {
2020-12-07 21:46:13 +00:00
/// Instantiate a new [`Server`] with all the default settings
pub fn new() -> Self {
Self {
timeout: Duration::from_secs(1),
complex_body_timeout_override: Some(Duration::from_secs(30)),
routes: RoutingNode::default(),
2020-12-15 15:32:31 +00:00
#[cfg(feature = "scgi_srv")]
autorewrite: false,
#[cfg(feature = "gemini_srv")]
cert_path: PathBuf::from("cert/cert.pem"),
#[cfg(feature = "gemini_srv")]
key_path: PathBuf::from("cert/key.pem"),
#[cfg(feature="ratelimiting")]
rate_limits: RoutingNode::default(),
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
data_dir: "data".into(),
#[cfg(feature="user_management")]
database: None,
#[cfg(feature="certgen")]
certgen_mode: CertGenMode::Interactive,
}
}
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
/// Sets the directory to store user data in
///
/// This will only be used if a database is not provided with [`set_database()`].
///
2020-11-16 06:13:16 +00:00
/// Defaults to `./data` if not specified
///
/// [`set_database()`]: Self::set_database()
2020-11-16 06:13:16 +00:00
pub fn set_database_dir(mut self, path: impl Into<PathBuf>) -> Self {
self.data_dir = path.into();
self
2020-10-31 19:53:03 +00:00
}
#[cfg(feature="user_management")]
/// Sets a specific database to use
///
/// This opens to trees within the database, both namespaced to avoid collisions.
///
/// If this is not provided, a database will be opened at the directory provided by
/// [`set_database_dir()`]
///
/// [`set_database_dir()`]: Self::set_database_dir()
pub fn set_database(mut self, db: sled::Db) -> Self {
self.database = Some(db);
self
}
#[cfg(feature="certgen")]
/// Determine where certificate config comes from, if generation is required
///
/// If a certificate & keyfile are not available on the provided path, they can be
/// generated. If this happens, several modes can be used to generate them, including
/// not generating them and just erroring, interactively prompting the user for
/// information, and using pre-provided information.
pub fn set_certificate_generation_mode(mut self, mode: CertGenMode) -> Self {
self.certgen_mode = mode;
self
}
#[cfg(feature = "gemini_srv")]
/// Sets the directory that kochab should look for TLS certs and keys into
///
/// Northstar will look for files called `cert.pem` and `key.pem` in the provided
/// directory.
///
/// This does not need to be set if both [`set_cert()`](Self::set_cert()) and
/// [`set_key()`](Self::set_key()) have been called.
///
/// If not set, the default is `cert/`
pub fn set_tls_dir(self, dir: impl Into<PathBuf>) -> Self {
let dir = dir.into();
self.set_cert(dir.join("cert.pem"))
.set_key(dir.join("key.pem"))
}
#[cfg(feature = "gemini_srv")]
/// Set the path to the TLS certificate kochab will use
///
/// This defaults to `cert/cert.pem`.
///
/// This does not need to be called it [`set_tls_dir()`](Self::set_tls_dir()) has been
/// called.
pub fn set_cert(mut self, cert_path: impl Into<PathBuf>) -> Self {
self.cert_path = cert_path.into();
self
}
#[cfg(feature = "gemini_srv")]
/// Set the path to the ertificate key kochab will use
///
/// This defaults to `cert/key.pem`.
///
/// This does not need to be called it [`set_tls_dir()`](Self::set_tls_dir()) has been
/// called.
///
/// This should of course correspond to the key set in
/// [`set_cert()`](Self::set_cert())
pub fn set_key(mut self, key_path: impl Into<PathBuf>) -> Self {
self.key_path = key_path.into();
self
2020-10-31 19:53:03 +00:00
}
/// Set the timeout on incoming requests
///
/// Note that this timeout is applied twice, once for the delivery of the request, and
/// once for sending the client's response. This means that for a 1 second timeout,
/// the client will have 1 second to complete the TLS handshake and deliver a request
/// header, then your API will have as much time as it needs to handle the request,
/// before the client has another second to receive the response.
///
/// If you would like a timeout for your code itself, please use
2020-11-18 02:01:54 +00:00
/// [`tokio::time::Timeout`] to implement it internally.
///
/// **The default timeout is 1 second.** As somewhat of a workaround for
/// shortcomings of the specification, this timeout, and any timeout set using this
/// method, is overridden in special cases, specifically for MIME types outside of
/// `text/plain` and `text/gemini`, to be 30 seconds. If you would like to change or
/// prevent this, please see
/// [`override_complex_body_timeout`](Self::override_complex_body_timeout()).
pub fn set_timeout(mut self, timeout: Duration) -> Self {
self.timeout = timeout;
self
2020-10-31 19:53:03 +00:00
}
/// Override the timeout for complex body types
///
/// Many clients choose to handle body types which cannot be displayed by prompting
/// the user if they would like to download or open the request body. However, since
/// this prompt occurs in the middle of receiving a request, often the connection
/// times out before the end user is able to respond to the prompt.
///
/// As a workaround, it is possible to set an override on the request timeout in
/// specific conditions:
///
/// 1. **Only override the timeout for receiving the body of the request.** This will
/// not override the timeout on sending the request header, nor on receiving the
/// response header.
/// 2. **Only override the timeout for successful responses.** The only bodies which
/// have bodies are successful ones. In all other cases, there's no body to
/// timeout for
/// 3. **Only override the timeout for complex body types.** Almost all clients are
/// able to display `text/plain` and `text/gemini` responses, and will not prompt
/// the user for these response types. This means that there is no reason to
/// expect a client to have a human-length response time for these MIME types.
/// Because of this, responses of this type will not be overridden.
///
/// This method is used to override the timeout for responses meeting these specific
/// criteria. All other stages of the connection will use the timeout specified in
/// [`set_timeout()`](Self::set_timeout()).
///
/// If this is set to [`None`], then the client will have the default amount of time
/// to both receive the header and the body. If this is set to [`Some`], the client
/// will have the default amount of time to recieve the header, and an *additional*
/// alotment of time to recieve the body.
///
/// The default timeout for this is 30 seconds.
pub fn override_complex_body_timeout(mut self, timeout: Option<Duration>) -> Self {
self.complex_body_timeout_override = timeout;
self
}
/// Add a handler for a route
///
/// A route must be an absolute path, for example "/endpoint" or "/", but not
/// "endpoint". Entering a relative or malformed path will result in a panic.
///
/// For more information about routing mechanics, see the docs for [`RoutingNode`].
pub fn add_route(mut self, path: &'static str, handler: impl Into<Handler>) -> Self {
self.routes.add_route(path, handler.into());
self
}
#[cfg(feature="ratelimiting")]
/// Add a rate limit to a route
///
/// The server will allow at most `burst` connections to any endpoints under this
/// route in a period of `period`. All extra requests will recieve a `SLOW_DOWN`, and
/// not be sent to the handler.
///
/// A route must be an absolute path, for example "/endpoint" or "/", but not
/// "endpoint". Entering a relative or malformed path will result in a panic.
///
/// For more information about routing mechanics, see the docs for [`RoutingNode`].
pub fn ratelimit(mut self, path: &'static str, burst: usize, period: Duration) -> Self {
let limiter = RateLimiter::new(period, burst);
self.rate_limits.add_route(path, limiter);
self
}
2020-12-15 15:32:31 +00:00
#[cfg_attr(feature = "gemini_srv", allow(unused_mut), allow(unused_variables))]
/// Enable or disable autorewrite
///
/// Many times, an app will served alongside other apps all on one domain. For
/// example:
/// * `gemini://example.com/gemlog/` might be some static content from a gemlog
/// handled by the gemini server
/// * `gemini://example.com/app/` might be where an SCGI app is hosted
/// * `gemini://example.com/` might be some static landing page linking to both the
/// gemlog and the app
///
/// If the user sent a request to `/app/path` in this case, the app would see it as
/// `/path` automatically, so the app doesn't need to care if it's mounted on `/` or
/// `/app`, because it can handle any request to `/path` the same.
///
/// The problem comes when the app needs to write a link. If an app were to send a
/// link to `/path`, expecting the user to come back with a request to `/path`, it
/// might be surprised to see that the user never arrives, and the user might be
/// surprised to find a `51 Not Found` error page.
///
/// This happens because when the user clicks the link to `/path`, their client takes
/// them to `gemini://example.com/path`. The gemini server doesn't see any apps or
/// files being served on `/path`, so it sends a `NotFound`.
///
/// The app *should* have linked to `/app/path`, but in order to do that, it would
/// need to know that it was mounted at `/app`, and include a bunch of logic to figure
/// out the write path. Thankfully, kochab can take care of this for you.
///
/// There are three main tools at your disposal for link rewriting:
///
/// * [`Server::set_autorewrite()`] is the easiest tool to use, and the one that will
/// work the best for most people. By setting this option on your server, it will
/// automatcially check for any gemini links in it's response before it's sent, and
/// rewrite them to be appropriate relative to the app. In this case, our example
/// app would simply send the link as `/path`, and kochab would catch and rewrite it
/// before it's sent out. For more information about this method, keep reading this
/// method's docs.
/// * [`Response::rewrite_all()`] will attempt to rewrite any links it finds in a
/// single response. This is the method that underlies [`Server::set_autorewrite()`],
/// but by calling it on your responses manually, you can choose exactly what
/// responses are rewritten.
/// * [`Request::rewrite_path()`] will rewrite a single link. This method works best
/// for when you need a lot of precision, like including links that need to be
/// rewritten alongside links that don't, or when you're rewriting links in
/// responses that aren't `text/gemini`.
///
/// All of these methods work on both `scgi_srv` and `gemini_srv` modes, so you can
/// use them regardless of what you plan on compiling your server to, which is
/// recommended if you're planning to offer compilation to either, or if you would
/// like to be able to change later. It's worth noting that while it will *work*, on
/// `gemini_srv` mode, `gemini_srv` servers are *always* mounted at the base path, so
/// this method really won't do anything other than sign off that the link is good.
///
/// If there's a problem with rewriting the URLs, typically because the proxy
/// server/SCGI client being used doesn't correctly implement the SCGI spec, then any
/// `text/gemini` responses bearing an absolute link will be `40 TEMPORARY FAILURE`,
/// and an error will be logged explaining what went wrong.
///
/// For more information about how rewritten paths are calculated, see
/// [`Request::rewrite_path()`].\
/// For more information about what responses are rewritten,
/// see [`Response::rewrite_all()`].
pub fn set_autorewrite(mut self, autorewrite: bool) -> Self {
2020-12-15 15:32:31 +00:00
#[cfg(feature = "scgi_srv")] {
self.autorewrite = autorewrite;
}
self
}
fn build(mut self) -> Result<ServerInner> {
#[cfg(feature = "gemini_srv")]
let config = cert::tls_config(
&self.cert_path,
&self.key_path,
#[cfg(feature="certgen")]
self.certgen_mode
).context("Failed to create TLS config")?;
2020-11-14 02:56:50 +00:00
self.routes.shrink();
#[cfg(feature="user_management")]
let data_dir = self.data_dir;
Ok(ServerInner {
routes: self.routes,
timeout: self.timeout,
complex_timeout: self.complex_body_timeout_override,
2020-12-15 15:32:31 +00:00
#[cfg(feature = "scgi_srv")]
autorewrite: self.autorewrite,
#[cfg(feature = "gemini_srv")]
tls_acceptor: TlsAcceptor::from(config),
#[cfg(feature="ratelimiting")]
rate_limits: self.rate_limits,
2020-11-16 06:13:16 +00:00
#[cfg(feature="user_management")]
manager: UserManager::new(
self.database.unwrap_or_else(move|| sled::open(data_dir).unwrap())
)?,
})
}
2020-11-13 19:20:59 +00:00
/// Start serving requests on a given bound address & port
///
/// `addr` can be anything `tokio` can parse, including just a string like
/// "localhost:1965"
///
/// This will only ever exit with an error. It's important to note that even if the
/// function exits, the server will NOT be deallocated, since references to it in
/// concurrently running futures may still exist. As such, a loop that handles an
/// error by re-serving a new server may trigger a memory leak.
pub async fn serve_ip(self, addr: impl ToSocketAddrs + Send) -> Result<()> {
let server = self.build()?;
let socket = TcpListener::bind(addr).await?;
server.serve_ip(socket).await
}
#[cfg(feature = "scgi_srv")]
/// Start serving requests on a given unix socket
///
/// Requires an address in the form of a path to bind to. This is only available when
/// in `scgi_srv` mode.
///
/// Please read the details and warnings of [`serve_ip()`] for more information
pub async fn serve_unix(self, addr: impl AsRef<std::path::Path>) -> Result<()> {
let server = self.build()?;
let socket = UnixListener::bind(addr)?;
server.serve_unix(socket).await
}
}
impl Default for Server {
fn default() -> Self {
Self::new()
2020-10-31 19:53:03 +00:00
}
}
async fn send_response_header(response: &Response, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
let meta = if response.meta.len() > 1024 {
warn!("Attempted to send response with META exceeding maximum length, truncating");
&response.meta[..1024]
} else {
&response.meta[..]
};
2020-10-31 19:53:03 +00:00
let header = format!(
"{status} {meta}\r\n",
status = response.status,
meta = meta,
2020-10-31 19:53:03 +00:00
);
stream.write_all(header.as_bytes()).await?;
2020-11-19 18:29:17 +00:00
stream.flush().await?;
Ok(())
}
async fn send_response_body(mut body: Option<Body>, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
match &mut body {
2020-12-02 00:10:50 +00:00
Some(Body::Bytes(ref bytes)) => stream.write_all(bytes).await?,
Some(Body::Reader(ref mut reader)) => { io::copy(reader, stream).await?; },
None => {},
2020-11-19 18:29:17 +00:00
}
2020-10-31 19:53:03 +00:00
if body.is_some() {
stream.flush().await?;
2020-10-31 19:53:03 +00:00
}
Ok(())
}
#[cfg(feature="ratelimiting")]
/// Every 5 minutes, remove excess keys from all ratelimiters
async fn prune_ratelimit_log(rate_limits: &'static RoutingNode<RateLimiter<IpAddr>>) -> Never {
let mut interval = interval(tokio::time::Duration::from_secs(10));
loop {
interval.tick().await;
rate_limits.iter().for_each(RateLimiter::trim_keys_verbose);
}
}
#[cfg(feature = "ratelimiting")]
enum Never {}
2020-12-03 20:04:12 +00:00
async fn opt_timeout<T>(duration: Option<time::Duration>, future: impl Future<Output = T>) -> Result<T, time::error::Elapsed> {
match duration {
Some(duration) => time::timeout(duration, future).await,
None => Ok(future.await),
}
}