1058 lines
42 KiB
Rust
1058 lines
42 KiB
Rust
#![warn(missing_docs)]
|
|
#![doc(html_logo_url = "https://raw.githubusercontent.com/twitter/twemoji/master/assets/svg/1f30c.svg")]
|
|
//! Kochab is an ergonomic and intuitive library for quickly building highly functional
|
|
//! and advanced Gemini applications on either SCGI or raw Gemini.
|
|
//!
|
|
//! Originally based on [northstar](https://crates.io/crates/northstar), though now far
|
|
//! diverged, kochab offers many features to help you build your application without any
|
|
//! of the boilerplate or counterintuitive shenanigans. Kochab centers around it's many
|
|
//! feature flags, which let you pick out exactly the features you need to build your
|
|
//! library, while leaving out features you don't need to keep your application
|
|
//! bloat-free.
|
|
//!
|
|
//! Another central feature of kochab is its multi-protocol abstraction. An application
|
|
//! built using kochab can easily compile either as a gemini application or as a SCGI
|
|
//! script using only a single feature flag.
|
|
//!
|
|
//! ## Features
|
|
//!
|
|
//! Kochab offers a wide array of features, so don't get overwhelmed. By default, you
|
|
//! start off with only the `gemini_srv` feature, and you're able to add on more features
|
|
//! as you need them. All of kochab's features are documented below.
|
|
//!
|
|
//! * `ratelimiting` - The ratelimiting feature adds in the ability to limit how often
|
|
//! users can access certain areas of an application. This is primarily configured using
|
|
//! the [`Server::ratelimit()`] method.
|
|
//!
|
|
//! * `serve_dir` - Adds in utilities for serving files & directories from the disk at
|
|
//! runtime. The easiest way to use this is to pass a [`PathBuf`] to the
|
|
//! [`Server::add_route()`] method, which will either serve a directory or a single file.
|
|
//! Files and directories can also be served using the methods in the [`util`] module.
|
|
//!
|
|
//! * `user_management` - Adds in tools to manage users using a certificate authentication
|
|
//! system. The user management suite is one of kocab's biggest features. When active,
|
|
//! kochab will maintain a database of registered users, linking each to a certificate.
|
|
//! Users also have custom data associated with them, which can be retrieved and modified
|
|
//! by the application.
|
|
//!
|
|
//! * `user_management_advanced` - Allows users to set a password and add additional
|
|
//! certificates. Without this feature, one certificate can only have one linked account,
|
|
//! unless you manually implement an authentication system. With this feature, kochab
|
|
//! will use argon2 to hash and check user passwords, and store user passwords alongside
|
|
//! the other user data
|
|
//!
|
|
//! * `user_management_routes` - The user management routes feature automates much of the
|
|
//! hard work of connecting the tools provided with the other two features with endpoints
|
|
//! that the user connects to. Kochab will manage all requests to the `/account` route,
|
|
//! and create pages to allow users to create an account, link new certificates, or
|
|
//! change/set their password. This also adds the ability to set an authenticated route,
|
|
//! which will automatically prompt the user to sign in, and give your handler access to
|
|
//! the user's data with no added work. This can be used with either the
|
|
//! `user_management_advanced` feature, or just the basic `user_management` feature
|
|
//!
|
|
//! * `certgen` - Enables automatically generating TLS certificates. Since only servers
|
|
//! directly using the gemini protocol need TLS certificates, this implies `gemini_srv`,
|
|
//! and should not be used with `scgi_srv`. By default, kochab will try to generate a
|
|
//! certificate by prompting the user in stdin/stdout, but this behavior can be customized
|
|
//! using [`Server::set_certificate_generation_mode()`].
|
|
//!
|
|
//! * `dashmap` - Enables some minor optimizations within the `user_management_routes`
|
|
//! feature. Automatically enabled by `ratelimiting`.
|
|
//!
|
|
//! * `ring` - When using `user_management_advanced` with `scgi_srv`, salts are calculated
|
|
//! based off of a simple PRNG and the system time. This should be plenty secure enough,
|
|
//! especially since we're using a good number argon2 rounds, but for bonus paranoia
|
|
//! points, you can add ring as a dependency to source secure random. This is enabled
|
|
//! automatically on `gemini_srv`, since `ring` is added as a dependency for certificate
|
|
//! processing. When not using the `user_management_advanced` feature, this does nothing
|
|
//! but increase your build time & size.
|
|
//!
|
|
//! * `gemini_srv`/`scgi_srv` - Switches between serving content using SCGI and serving
|
|
//! content as a raw gemini server. One and only one of these features must be enabled,
|
|
//! and compilation will fail if both are enabled. See below for more information.
|
|
//!
|
|
//! ## Gemini & SCGI Modes
|
|
//!
|
|
//! **It is highly recommended that you read this section, *especially* if you don't know
|
|
//! what SCGI is.**
|
|
//!
|
|
//! Central to kochab's repertoire is the ability to serve either content either through
|
|
//! raw Gemini or through a reverse proxy with SCGI. This can be accomplished easily
|
|
//! using a single feature flag. But first:
|
|
//!
|
|
//! ### What's SCGI and why should I be using it?
|
|
//!
|
|
//! You're probably familiar with the Gemini protocol, and how servers serve things on it.
|
|
//! Gemini is easy to serve and simple to work with. You probably went into this project
|
|
//! expecting to serve content directly through the Gemini protocol. So what's this about
|
|
//! SCGI, and why should you bother using it?
|
|
//!
|
|
//! The problem that SCGI solves is that it is very difficult to have multiple servers on
|
|
//! one domain with Gemini. For example, if you wanted to serve your blog on `/blog.gmi`,
|
|
//! but wanted to have an app running on `/app`, you would most likely have to use SCGI.
|
|
//!
|
|
//! SCGI has two parts: A main gemini server (called the SCGI client) that handles TLS
|
|
//! for incoming connections and either serves some static content, runs some other
|
|
//! handler, or passes it off to the SCGI server, which is what you'd be writing. The
|
|
//! SCGI server doesn't directly interact with the client, but gets plenty of information
|
|
//! about the connection from the server, like what path the server is being served on
|
|
//! (like "/app"), the end user's IP, and the certificate fingerprint being used, if
|
|
//! applicable.
|
|
//!
|
|
//! Because SCGI servers don't need to bother with TLS, compiling your app as SCGI will
|
|
//! also be fairly faster, since kochab doesn't need to bring in `rustls` or any
|
|
//! TLS certificate generation libraries.
|
|
//!
|
|
//! This doesn't necessarily mean that SCGI is for every app, but if you suspect that a
|
|
//! user might ever need to run your app on a path other than the main one, or may want to
|
|
//! serve static files alongside your site, I encourage you to seriously consider it.
|
|
//!
|
|
//! ### But I don't want to bother learning a new protocol ![blobcat-pout]
|
|
//!
|
|
//! You don't have to! Kochab handles everything about it, thanks to the magic of
|
|
//! abstraction! The only thing you have to change are the feature flags in your
|
|
//! `Config.toml`. In fact, you could even expose the feature flag so that users can
|
|
//! compile your crate as either a Gemini or SCGI server without needing to write any
|
|
//! conditional compilation!
|
|
//!
|
|
//! ### Getting started
|
|
//!
|
|
//! **Updating your feature flags**
|
|
//!
|
|
//! By default, Kochab serves content over raw gemini, to make it easier for new users to
|
|
//! jump right into using the library. This is done using the on-by-default `gemini_srv`
|
|
//! feature flag. To switch to SCGI, we want to switch off `gemini_srv` and switch on
|
|
//! `scgi_srv`.
|
|
//!
|
|
//! ```toml
|
|
//! [dependencies.kochab]
|
|
//! git = "https://gitlab.com/Alch_Emi/kochab.git"
|
|
//! branch = "stable"
|
|
//! default-features = false
|
|
//! features = ["scgi_srv"] # and any other features you might need
|
|
//! ```
|
|
//!
|
|
//! **Testing with a minimal SCGI client**
|
|
//!
|
|
//! To give your code a run, you'll need a server to handle Gemini requests and pass them
|
|
//! off to your SCGI server. There's a few Gemini servers out there with SCGI support,
|
|
//! but if you're just interested in giving your code a quick run, I'd recommend
|
|
//! mollybrown, which has very good SCGI support and is super easy to set up
|
|
//!
|
|
//! You can grab a copy of molly brown from [tildegit.org/solderpunk/molly-brown][1].
|
|
//!
|
|
//! Once you have it, you can find a super simple configuration file [here][2], and then
|
|
//! just run
|
|
//!
|
|
//! ```sh
|
|
//! molly-brown -c molly-brown.conf
|
|
//! ```
|
|
//!
|
|
//! Now, when you run your code, you can connect to `localhost`, and molly brown will
|
|
//! connect to your SCGI server and forward the response on to your Gemini client.
|
|
//!
|
|
//! **Rewriting Paths**
|
|
//!
|
|
//! One important difference about writing code for an SCGI server is that your app might
|
|
//! be served on a path that's not the base path. If this happens, you suddenly have a
|
|
//! distinction between an absolute link for your app, and an absolute link for the gemini
|
|
//! server.
|
|
//!
|
|
//! For example, if an app that's being served on `/app` has a link to `/path`, this could
|
|
//! be either:
|
|
//!
|
|
//! * Meant to be handled by the app by the route at `/path`, which would be `/app/path`
|
|
//! for the parent Gemini server, or
|
|
//! * Meant to link to some content on the parent gemini server, at `/path`, which would
|
|
//! mean linking to a url your app doesn't control
|
|
//!
|
|
//! Most of the time, you want to do the first one. Thankfully, Kochab makes rewriting
|
|
//! links relative to the base of your app super easy. In most cases, all you need is to
|
|
//! add a single line to your Server builder pattern.
|
|
//!
|
|
//! For more information, see [`Server::set_autorewrite()`].
|
|
//!
|
|
//! [1]: https://tildegit.org/solderpunk/molly-brown
|
|
//! [2]: https://gitlab.com/Alch_Emi/kochab/-/raw/244fd251/molly-brown.conf
|
|
//! [blobcat-pout]: https://the-apothecary.club/_matrix/media/r0/thumbnail/the-apothecary.club/10a406405a5bcd699a5328259133bfd9260320a6?height=99&width=20 ":blobcat-pout:"
|
|
//! <style>
|
|
//! img[alt=blobcat-pout] { width: 20px; vertical-align: top; }
|
|
//! </style>
|
|
|
|
#[macro_use] extern crate log;
|
|
|
|
#[cfg(all(feature = "gemini_srv", feature = "scgi_srv", not(doc)))]
|
|
compile_error!("Please enable only one of either the `gemini_srv` or `scgi_srv` features on the kochab crate");
|
|
|
|
#[cfg(not(any(feature = "gemini_srv", feature = "scgi_srv")))]
|
|
compile_error!("Please enable at least one of either the `gemini_srv` or `scgi_srv` features on the kochab crate");
|
|
|
|
use std::{
|
|
sync::Arc,
|
|
time::Duration,
|
|
future::Future,
|
|
};
|
|
#[cfg(feature = "gemini_srv")]
|
|
use std::convert::TryFrom;
|
|
#[cfg(feature = "scgi_srv")]
|
|
use std::{
|
|
collections::HashMap,
|
|
net::SocketAddr,
|
|
str::FromStr,
|
|
};
|
|
#[cfg(any(feature = "gemini_srv", feature = "user_management"))]
|
|
use std::path::PathBuf;
|
|
#[cfg(feature="ratelimiting")]
|
|
use std::net::IpAddr;
|
|
use tokio::{
|
|
io,
|
|
io::BufReader,
|
|
net::TcpListener,
|
|
net::ToSocketAddrs,
|
|
time,
|
|
prelude::*,
|
|
};
|
|
#[cfg(feature = "scgi_srv")]
|
|
use tokio::net::UnixListener;
|
|
#[cfg(feature = "gemini_srv")]
|
|
use tokio::{
|
|
time::timeout,
|
|
net::TcpStream,
|
|
};
|
|
#[cfg(feature = "ratelimiting")]
|
|
use tokio::time::interval;
|
|
use anyhow::*;
|
|
use routing::RoutingNode;
|
|
#[cfg(feature = "ratelimiting")]
|
|
use ratelimiting::RateLimiter;
|
|
#[cfg(feature = "gemini_srv")]
|
|
use tokio_rustls::TlsAcceptor;
|
|
#[cfg(feature = "gemini_srv")]
|
|
use rustls::Session;
|
|
|
|
mod types;
|
|
mod handling;
|
|
pub mod util;
|
|
pub mod routing;
|
|
#[cfg(feature = "ratelimiting")]
|
|
mod ratelimiting;
|
|
#[cfg(feature = "user_management")]
|
|
pub mod user_management;
|
|
#[cfg(feature = "gemini_srv")]
|
|
mod cert;
|
|
|
|
#[cfg(feature="user_management")]
|
|
use user_management::UserManager;
|
|
#[cfg(feature = "certgen")]
|
|
pub use cert::CertGenMode;
|
|
|
|
pub use uriparse::URIReference;
|
|
pub use types::*;
|
|
pub use handling::Handler;
|
|
|
|
/// The maximun length of a Request URI
|
|
pub const REQUEST_URI_MAX_LEN: usize = 1024;
|
|
|
|
/// The default port for the gemini protocol
|
|
pub const GEMINI_PORT: u16 = 1965;
|
|
|
|
#[derive(Clone)]
|
|
struct ServerInner {
|
|
#[cfg(feature = "gemini_srv")]
|
|
tls_acceptor: TlsAcceptor,
|
|
routes: Arc<RoutingNode<Handler>>,
|
|
timeout: Duration,
|
|
complex_timeout: Option<Duration>,
|
|
autorewrite: bool,
|
|
#[cfg(feature="ratelimiting")]
|
|
rate_limits: Arc<RoutingNode<RateLimiter<IpAddr>>>,
|
|
#[cfg(feature="user_management")]
|
|
manager: UserManager,
|
|
}
|
|
|
|
impl ServerInner {
|
|
async fn serve_ip(self, listener: TcpListener) -> Result<()> {
|
|
#[cfg(feature = "ratelimiting")]
|
|
tokio::spawn(prune_ratelimit_log(self.rate_limits.clone()));
|
|
|
|
loop {
|
|
let (stream, _addr) = listener.accept().await
|
|
.context("Failed to accept client")?;
|
|
let this = self.clone();
|
|
|
|
tokio::spawn(async move {
|
|
if let Err(err) = this.serve_client(stream).await {
|
|
error!("{:?}", err);
|
|
}
|
|
});
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
// Yeah it's code duplication, but I can't find a way around it, so this is what we're
|
|
// getting for now
|
|
async fn serve_unix(self, listener: UnixListener) -> Result<()> {
|
|
#[cfg(feature = "ratelimiting")]
|
|
tokio::spawn(prune_ratelimit_log(self.rate_limits.clone()));
|
|
|
|
loop {
|
|
let (stream, _addr) = listener.accept().await
|
|
.context("Failed to accept client")?;
|
|
let this = self.clone();
|
|
|
|
tokio::spawn(async move {
|
|
if let Err(err) = this.serve_client(stream).await {
|
|
error!("{:?}", err);
|
|
}
|
|
});
|
|
}
|
|
}
|
|
|
|
async fn serve_client(
|
|
&self,
|
|
#[cfg(feature = "gemini_srv")]
|
|
stream: TcpStream,
|
|
#[cfg(all(feature = "scgi_srv", not(feature = "gemini_srv")))]
|
|
stream: impl AsyncWrite + AsyncRead + Unpin + Send,
|
|
) -> Result<()> {
|
|
let fut_accept_request = async {
|
|
#[cfg(feature = "gemini_srv")]
|
|
let stream = self.tls_acceptor.accept(stream).await
|
|
.context("Failed to establish TLS session")?;
|
|
let mut stream = BufReader::new(stream);
|
|
|
|
let request = self.receive_request(&mut stream).await
|
|
.context("Failed to receive request")?;
|
|
|
|
Result::<_, anyhow::Error>::Ok((request, stream))
|
|
};
|
|
|
|
|
|
// Wait for the request to be parsed
|
|
let (mut request, mut stream) = {
|
|
#[cfg(feature = "gemini_srv")] {
|
|
// Use a timeout for interacting with the client
|
|
let fut_accept_request = timeout(self.timeout, fut_accept_request);
|
|
fut_accept_request.await
|
|
.context("Client timed out while waiting for response")??
|
|
}
|
|
#[cfg(feature = "scgi_srv")]
|
|
fut_accept_request.await?
|
|
};
|
|
|
|
// Determine the remote client's IP address for logging and ratelimiting
|
|
let peer_addr = {
|
|
#[cfg(feature = "gemini_srv")] {
|
|
stream.get_ref()
|
|
.get_ref()
|
|
.0
|
|
.peer_addr()?
|
|
.ip()
|
|
}
|
|
#[cfg(feature = "scgi_srv")] {
|
|
let remote = request.headers()
|
|
.get("REMOTE_ADDR")
|
|
.ok_or(ParseError::Malformed("REMOTE_ADDR header not received"))?
|
|
.as_str();
|
|
SocketAddr::from_str(remote)
|
|
.map(|a| a.ip())
|
|
.or_else(|_| std::net::IpAddr::from_str(remote))
|
|
.context("Received malformed IP address from upstream")?
|
|
}
|
|
};
|
|
|
|
#[cfg(feature = "ratelimiting")]
|
|
// Perform ratelimiting checks
|
|
if let Some(resp) = self.check_rate_limits(peer_addr, &request) {
|
|
|
|
// Log warning
|
|
warn!(
|
|
"Client from {} requesting {} was turned away by ratelimiting",
|
|
peer_addr,
|
|
request.uri()
|
|
);
|
|
|
|
// Send error response
|
|
self.send_response(resp, &mut stream).await
|
|
.context("Failed to send response")?;
|
|
|
|
// Exit
|
|
return Ok(())
|
|
}
|
|
|
|
info!("{} requested: {}", peer_addr, request.uri());
|
|
|
|
// Identify the client certificate from the tls stream. This is the first
|
|
// certificate in the certificate chain.
|
|
#[cfg(feature = "gemini_srv")] { // This is done earlier for `scgi_srv`
|
|
let client_cert = stream.get_ref()
|
|
.get_ref()
|
|
.1
|
|
.get_peer_certificates()
|
|
.and_then(|mut v| if v.is_empty() {None} else {Some(v.remove(0))});
|
|
|
|
request.set_cert(client_cert);
|
|
}
|
|
|
|
#[cfg_attr(feature = "gemini_srv", allow(unused_mut))] // Used for scgi_srv only
|
|
let mut response = if let Some((trailing, handler)) = self.routes.match_request(&request) {
|
|
request.set_trailing(trailing);
|
|
handler.handle(request.clone()).await
|
|
} else {
|
|
Response::not_found()
|
|
};
|
|
|
|
#[cfg(feature = "scgi_srv")] // No point running noop code
|
|
if self.autorewrite {
|
|
match response.rewrite_all(&request).await {
|
|
Ok(true) => { /* all is well */ }
|
|
Ok(false) => {
|
|
error!(
|
|
concat!(
|
|
"Upstream did not include SCRIPT_PATH or SCRIPT_NAME, refusing to",
|
|
" serve any text/gemini content with absolute links. It's most",
|
|
" likely that the proxy server you're using doesn't correctly",
|
|
" or completely implement the SCGI specification.",
|
|
)
|
|
);
|
|
response = Response::temporary_failure("Server misconfigured");
|
|
},
|
|
Err(e) => {
|
|
error!("Error reading text/gemini file from Response reader: {}", e);
|
|
response = Response::not_found();
|
|
}
|
|
}
|
|
}
|
|
|
|
self.send_response(response, &mut stream).await
|
|
.context("Failed to send response")?;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
async fn send_response(&self, response: Response, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
|
|
let use_complex_timeout =
|
|
response.body.is_some() &&
|
|
response.meta != "text/plain" &&
|
|
response.meta != "text/gemini" &&
|
|
self.complex_timeout.is_some();
|
|
|
|
let send_general_timeout;
|
|
let send_header_timeout;
|
|
let send_body_timeout;
|
|
|
|
if use_complex_timeout {
|
|
send_general_timeout = None;
|
|
send_header_timeout = Some(self.timeout);
|
|
send_body_timeout = self.complex_timeout;
|
|
} else {
|
|
send_general_timeout = Some(self.timeout);
|
|
send_header_timeout = None;
|
|
send_body_timeout = None;
|
|
}
|
|
|
|
if !response.is_success() && response.body.is_some() {
|
|
warn!(concat!(
|
|
"Received a response with a body, but a status code of {} (!= 20). ",
|
|
" Responses should only have a body if their status code is 20. The body",
|
|
" will be sent, but this is likely to cause unexpected behavior."),
|
|
response.status,
|
|
);
|
|
}
|
|
|
|
opt_timeout(send_general_timeout, async {
|
|
// Send the header
|
|
opt_timeout(send_header_timeout, send_response_header(&response, stream))
|
|
.await
|
|
.context("Timed out while sending response header")?
|
|
.context("Failed to write response header")?;
|
|
|
|
// Send the body
|
|
opt_timeout(send_body_timeout, send_response_body(response.body, stream))
|
|
.await
|
|
.context("Timed out while sending response body")?
|
|
.context("Failed to write response body")?;
|
|
|
|
Ok::<_,Error>(())
|
|
})
|
|
.await
|
|
.context("Timed out while sending response data")??;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
#[cfg(feature="ratelimiting")]
|
|
fn check_rate_limits(&self, addr: IpAddr, req: &Request) -> Option<Response> {
|
|
if let Some((_, limiter)) = self.rate_limits.match_request(req) {
|
|
if let Err(when) = limiter.check_key(addr) {
|
|
return Some(Response::slow_down(when.as_secs()))
|
|
}
|
|
}
|
|
None
|
|
}
|
|
|
|
#[cfg(feature = "gemini_srv")]
|
|
async fn receive_request(
|
|
&self,
|
|
stream: &mut (impl AsyncBufRead + Unpin + Send),
|
|
) -> Result<Request> {
|
|
const HEADER_LIMIT: usize = REQUEST_URI_MAX_LEN + "\r\n".len();
|
|
let mut stream = stream.take(HEADER_LIMIT as u64);
|
|
let mut uri = Vec::new();
|
|
|
|
stream.read_until(b'\n', &mut uri).await?;
|
|
|
|
if !uri.ends_with(b"\r\n") {
|
|
if uri.len() < REQUEST_URI_MAX_LEN {
|
|
bail!("Request header not terminated with CRLF")
|
|
} else {
|
|
bail!("Request URI too long")
|
|
}
|
|
}
|
|
|
|
// Strip CRLF
|
|
uri.pop();
|
|
uri.pop();
|
|
|
|
let uri = URIReference::try_from(&*uri)
|
|
.context("Request URI is invalid")?
|
|
.into_owned();
|
|
|
|
Request::new(
|
|
uri,
|
|
#[cfg(feature="user_management")]
|
|
self.manager.clone(),
|
|
).context("Failed to create request from URI")
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
async fn receive_request(
|
|
&self,
|
|
stream: &mut (impl AsyncBufRead + Unpin),
|
|
) -> Result<Request> {
|
|
let mut buff = Vec::with_capacity(4);
|
|
|
|
#[allow(clippy::char_lit_as_u8)]
|
|
// Read the length of the header netstring (e.g. "120:")
|
|
stream.read_until(':' as u8, &mut buff).await?;
|
|
|
|
buff.pop(); // Remove the trailing ':'
|
|
let len = std::str::from_utf8(&*buff)
|
|
.ok()
|
|
.and_then(|s| usize::from_str(s).ok())
|
|
.ok_or(ParseError::Malformed("netstring length"))?;
|
|
|
|
// Read in the headers
|
|
buff.clear();
|
|
buff.resize(len + 1, 0);
|
|
stream.read_exact(buff.as_mut()).await?;
|
|
buff.truncate(len - 1); // Remove the final \x00,
|
|
|
|
// Parse the headers
|
|
let (maybe_trailing, headers) = buff.split(|b| *b == 0) // Headers are null delimiited
|
|
.map(|bytes| // Convert to an &str
|
|
std::str::from_utf8(bytes)
|
|
.map_err(|_| ParseError::Malformed("scgi headers"))
|
|
.map(str::trim)
|
|
)
|
|
.try_fold( // Turn the array of [header, value, header, ...] into a map
|
|
(Option::<&str>::None, HashMap::<String, String>::with_capacity(16)),
|
|
|(last_header, mut headers), s| {
|
|
s.map(|text| {
|
|
match last_header {
|
|
None => (Some(text), headers),
|
|
Some(header) => {
|
|
headers.insert(header.to_string(), text.to_string());
|
|
(None, headers)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
)?;
|
|
|
|
// If there's not the same number of headers as values, that's a problem
|
|
if maybe_trailing.is_some() {
|
|
bail!(ParseError::Malformed("trailing header"));
|
|
}
|
|
|
|
// Check the content length info
|
|
let cont_len_val = headers.get("CONTENT_LENGTH")
|
|
.ok_or(ParseError::Malformed("No content length header!"))?;
|
|
let cont_len = usize::from_str(cont_len_val)
|
|
.map_err(|_| ParseError::Malformed("Malformed content length"))?;
|
|
if cont_len > 0 {
|
|
bail!(ParseError::Malformed("Gemini SCGI requests should not have a body"));
|
|
}
|
|
|
|
// Spec requires setting an SCGI header to one
|
|
if *headers.get("SCGI").ok_or(ParseError::Malformed("No SCGI header"))? != "1" {
|
|
bail!(ParseError::Malformed("SCGI header not set to \"1\""));
|
|
}
|
|
|
|
trace!("Headers received: {:?}", headers);
|
|
|
|
Ok(
|
|
Request::new(
|
|
headers,
|
|
#[cfg(feature = "user_management")]
|
|
self.manager.clone(),
|
|
)?
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
#[cfg(feature = "scgi_srv")]
|
|
enum ParseError {
|
|
IO(io::Error),
|
|
Malformed(&'static str),
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
impl From<io::Error> for ParseError {
|
|
fn from(e: io::Error) -> Self {
|
|
Self::IO(e)
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
impl std::fmt::Display for ParseError {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
|
match self {
|
|
Self::IO(e) => write!(f, "IO Error while parsing and responding SCGI: {}", e),
|
|
Self::Malformed(e) => write!(f, "SCGI request malformed at {}", e),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
impl std::error::Error for ParseError {
|
|
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
|
|
if let Self::IO(e) = self { Some(e) } else { None }
|
|
}
|
|
}
|
|
|
|
/// A builder for configuring a kochab server
|
|
///
|
|
/// Once created with [`Server::new()`], different configuration methods can be
|
|
/// called to set up the server, before finally making a call to [`serve_ip()`] or
|
|
/// [`serve_unix()`].
|
|
///
|
|
/// Technically, no methods need to be called in order to create the server, but unless
|
|
/// you add at least one route with [`add_route()`], the server will respond with `51 NOT
|
|
/// FOUND` to all requests.
|
|
///
|
|
/// # Example
|
|
/// ```no_run
|
|
/// use anyhow::Result;
|
|
/// # use kochab::Response;
|
|
/// # use kochab::Request;
|
|
/// # use kochab::Server;
|
|
///
|
|
/// #[tokio::main]
|
|
/// async fn main() {
|
|
/// Server::new()
|
|
/// .add_route("/", hello_world)
|
|
/// .serve_ip("localhost:1965")
|
|
/// .await;
|
|
/// }
|
|
///
|
|
/// async fn hello_world(_: Request) -> Result<Response> {
|
|
/// Ok(Response::success_gemini("Hello world!"))
|
|
/// }
|
|
/// ```
|
|
///
|
|
/// [`serve_ip()`]: Self::serve_ip()
|
|
/// [`serve_unix()`]: Self::serve_unix()
|
|
/// [`add_route()`]: Self::add_route()
|
|
pub struct Server {
|
|
timeout: Duration,
|
|
complex_body_timeout_override: Option<Duration>,
|
|
routes: RoutingNode<Handler>,
|
|
autorewrite: bool,
|
|
#[cfg(feature = "gemini_srv")]
|
|
cert_path: PathBuf,
|
|
#[cfg(feature = "gemini_srv")]
|
|
key_path: PathBuf,
|
|
#[cfg(feature="ratelimiting")]
|
|
rate_limits: RoutingNode<RateLimiter<IpAddr>>,
|
|
#[cfg(feature="user_management")]
|
|
data_dir: PathBuf,
|
|
#[cfg(feature="user_management")]
|
|
database: Option<sled::Db>,
|
|
#[cfg(feature="certgen")]
|
|
certgen_mode: CertGenMode,
|
|
}
|
|
|
|
impl Server {
|
|
/// Instantiate a new [`Server`] with all the default settings
|
|
pub fn new() -> Self {
|
|
Self {
|
|
timeout: Duration::from_secs(1),
|
|
complex_body_timeout_override: Some(Duration::from_secs(30)),
|
|
routes: RoutingNode::default(),
|
|
autorewrite: false,
|
|
#[cfg(feature = "gemini_srv")]
|
|
cert_path: PathBuf::from("cert/cert.pem"),
|
|
#[cfg(feature = "gemini_srv")]
|
|
key_path: PathBuf::from("cert/key.pem"),
|
|
#[cfg(feature="ratelimiting")]
|
|
rate_limits: RoutingNode::default(),
|
|
#[cfg(feature="user_management")]
|
|
data_dir: "data".into(),
|
|
#[cfg(feature="user_management")]
|
|
database: None,
|
|
#[cfg(feature="certgen")]
|
|
certgen_mode: CertGenMode::Interactive,
|
|
}
|
|
}
|
|
|
|
#[cfg(feature="user_management")]
|
|
/// Sets the directory to store user data in
|
|
///
|
|
/// This will only be used if a database is not provided with [`set_database()`].
|
|
///
|
|
/// Defaults to `./data` if not specified
|
|
///
|
|
/// [`set_database()`]: Self::set_database()
|
|
pub fn set_database_dir(mut self, path: impl Into<PathBuf>) -> Self {
|
|
self.data_dir = path.into();
|
|
self
|
|
}
|
|
|
|
#[cfg(feature="user_management")]
|
|
/// Sets a specific database to use
|
|
///
|
|
/// This opens to trees within the database, both namespaced to avoid collisions.
|
|
///
|
|
/// If this is not provided, a database will be opened at the directory provided by
|
|
/// [`set_database_dir()`]
|
|
///
|
|
/// [`set_database_dir()`]: Self::set_database_dir()
|
|
pub fn set_database(mut self, db: sled::Db) -> Self {
|
|
self.database = Some(db);
|
|
self
|
|
}
|
|
|
|
#[cfg(feature="certgen")]
|
|
/// Determine where certificate config comes from, if generation is required
|
|
///
|
|
/// If a certificate & keyfile are not available on the provided path, they can be
|
|
/// generated. If this happens, several modes can be used to generate them, including
|
|
/// not generating them and just erroring, interactively prompting the user for
|
|
/// information, and using pre-provided information.
|
|
pub fn set_certificate_generation_mode(mut self, mode: CertGenMode) -> Self {
|
|
self.certgen_mode = mode;
|
|
self
|
|
}
|
|
|
|
#[cfg(feature = "gemini_srv")]
|
|
/// Sets the directory that kochab should look for TLS certs and keys into
|
|
///
|
|
/// Northstar will look for files called `cert.pem` and `key.pem` in the provided
|
|
/// directory.
|
|
///
|
|
/// This does not need to be set if both [`set_cert()`](Self::set_cert()) and
|
|
/// [`set_key()`](Self::set_key()) have been called.
|
|
///
|
|
/// If not set, the default is `cert/`
|
|
pub fn set_tls_dir(self, dir: impl Into<PathBuf>) -> Self {
|
|
let dir = dir.into();
|
|
self.set_cert(dir.join("cert.pem"))
|
|
.set_key(dir.join("key.pem"))
|
|
}
|
|
|
|
#[cfg(feature = "gemini_srv")]
|
|
/// Set the path to the TLS certificate kochab will use
|
|
///
|
|
/// This defaults to `cert/cert.pem`.
|
|
///
|
|
/// This does not need to be called it [`set_tls_dir()`](Self::set_tls_dir()) has been
|
|
/// called.
|
|
pub fn set_cert(mut self, cert_path: impl Into<PathBuf>) -> Self {
|
|
self.cert_path = cert_path.into();
|
|
self
|
|
}
|
|
|
|
#[cfg(feature = "gemini_srv")]
|
|
/// Set the path to the ertificate key kochab will use
|
|
///
|
|
/// This defaults to `cert/key.pem`.
|
|
///
|
|
/// This does not need to be called it [`set_tls_dir()`](Self::set_tls_dir()) has been
|
|
/// called.
|
|
///
|
|
/// This should of course correspond to the key set in
|
|
/// [`set_cert()`](Self::set_cert())
|
|
pub fn set_key(mut self, key_path: impl Into<PathBuf>) -> Self {
|
|
self.key_path = key_path.into();
|
|
self
|
|
}
|
|
|
|
/// Set the timeout on incoming requests
|
|
///
|
|
/// Note that this timeout is applied twice, once for the delivery of the request, and
|
|
/// once for sending the client's response. This means that for a 1 second timeout,
|
|
/// the client will have 1 second to complete the TLS handshake and deliver a request
|
|
/// header, then your API will have as much time as it needs to handle the request,
|
|
/// before the client has another second to receive the response.
|
|
///
|
|
/// If you would like a timeout for your code itself, please use
|
|
/// [`tokio::time::Timeout`] to implement it internally.
|
|
///
|
|
/// **The default timeout is 1 second.** As somewhat of a workaround for
|
|
/// shortcomings of the specification, this timeout, and any timeout set using this
|
|
/// method, is overridden in special cases, specifically for MIME types outside of
|
|
/// `text/plain` and `text/gemini`, to be 30 seconds. If you would like to change or
|
|
/// prevent this, please see
|
|
/// [`override_complex_body_timeout`](Self::override_complex_body_timeout()).
|
|
pub fn set_timeout(mut self, timeout: Duration) -> Self {
|
|
self.timeout = timeout;
|
|
self
|
|
}
|
|
|
|
/// Override the timeout for complex body types
|
|
///
|
|
/// Many clients choose to handle body types which cannot be displayed by prompting
|
|
/// the user if they would like to download or open the request body. However, since
|
|
/// this prompt occurs in the middle of receiving a request, often the connection
|
|
/// times out before the end user is able to respond to the prompt.
|
|
///
|
|
/// As a workaround, it is possible to set an override on the request timeout in
|
|
/// specific conditions:
|
|
///
|
|
/// 1. **Only override the timeout for receiving the body of the request.** This will
|
|
/// not override the timeout on sending the request header, nor on receiving the
|
|
/// response header.
|
|
/// 2. **Only override the timeout for successful responses.** The only bodies which
|
|
/// have bodies are successful ones. In all other cases, there's no body to
|
|
/// timeout for
|
|
/// 3. **Only override the timeout for complex body types.** Almost all clients are
|
|
/// able to display `text/plain` and `text/gemini` responses, and will not prompt
|
|
/// the user for these response types. This means that there is no reason to
|
|
/// expect a client to have a human-length response time for these MIME types.
|
|
/// Because of this, responses of this type will not be overridden.
|
|
///
|
|
/// This method is used to override the timeout for responses meeting these specific
|
|
/// criteria. All other stages of the connection will use the timeout specified in
|
|
/// [`set_timeout()`](Self::set_timeout()).
|
|
///
|
|
/// If this is set to [`None`], then the client will have the default amount of time
|
|
/// to both receive the header and the body. If this is set to [`Some`], the client
|
|
/// will have the default amount of time to recieve the header, and an *additional*
|
|
/// alotment of time to recieve the body.
|
|
///
|
|
/// The default timeout for this is 30 seconds.
|
|
pub fn override_complex_body_timeout(mut self, timeout: Option<Duration>) -> Self {
|
|
self.complex_body_timeout_override = timeout;
|
|
self
|
|
}
|
|
|
|
/// Add a handler for a route
|
|
///
|
|
/// A route must be an absolute path, for example "/endpoint" or "/", but not
|
|
/// "endpoint". Entering a relative or malformed path will result in a panic.
|
|
///
|
|
/// For more information about routing mechanics, see the docs for [`RoutingNode`].
|
|
pub fn add_route(mut self, path: &'static str, handler: impl Into<Handler>) -> Self {
|
|
self.routes.add_route(path, handler.into());
|
|
self
|
|
}
|
|
|
|
#[cfg(feature="ratelimiting")]
|
|
/// Add a rate limit to a route
|
|
///
|
|
/// The server will allow at most `burst` connections to any endpoints under this
|
|
/// route in a period of `period`. All extra requests will recieve a `SLOW_DOWN`, and
|
|
/// not be sent to the handler.
|
|
///
|
|
/// A route must be an absolute path, for example "/endpoint" or "/", but not
|
|
/// "endpoint". Entering a relative or malformed path will result in a panic.
|
|
///
|
|
/// For more information about routing mechanics, see the docs for [`RoutingNode`].
|
|
pub fn ratelimit(mut self, path: &'static str, burst: usize, period: Duration) -> Self {
|
|
let limiter = RateLimiter::new(period, burst);
|
|
self.rate_limits.add_route(path, limiter);
|
|
self
|
|
}
|
|
|
|
/// Enable or disable autorewrite
|
|
///
|
|
/// Many times, an app will served alongside other apps all on one domain. For
|
|
/// example:
|
|
/// * `gemini://example.com/gemlog/` might be some static content from a gemlog
|
|
/// handled by the gemini server
|
|
/// * `gemini://example.com/app/` might be where an SCGI app is hosted
|
|
/// * `gemini://example.com/` might be some static landing page linking to both the
|
|
/// gemlog and the app
|
|
///
|
|
/// If the user sent a request to `/app/path` in this case, the app would see it as
|
|
/// `/path` automatically, so the app doesn't need to care if it's mounted on `/` or
|
|
/// `/app`, because it can handle any request to `/path` the same.
|
|
///
|
|
/// The problem comes when the app needs to write a link. If an app were to send a
|
|
/// link to `/path`, expecting the user to come back with a request to `/path`, it
|
|
/// might be surprised to see that the user never arrives, and the user might be
|
|
/// surprised to find a `51 Not Found` error page.
|
|
///
|
|
/// This happens because when the user clicks the link to `/path`, their client takes
|
|
/// them to `gemini://example.com/path`. The gemini server doesn't see any apps or
|
|
/// files being served on `/path`, so it sends a `NotFound`.
|
|
///
|
|
/// The app *should* have linked to `/app/path`, but in order to do that, it would
|
|
/// need to know that it was mounted at `/app`, and include a bunch of logic to figure
|
|
/// out the write path. Thankfully, kochab can take care of this for you.
|
|
///
|
|
/// There are three main tools at your disposal for link rewriting:
|
|
///
|
|
/// * [`Server::set_autorewrite()`] is the easiest tool to use, and the one that will
|
|
/// work the best for most people. By setting this option on your server, it will
|
|
/// automatcially check for any gemini links in it's response before it's sent, and
|
|
/// rewrite them to be appropriate relative to the app. In this case, our example
|
|
/// app would simply send the link as `/path`, and kochab would catch and rewrite it
|
|
/// before it's sent out. For more information about this method, keep reading this
|
|
/// method's docs.
|
|
/// * [`Response::rewrite_all()`] will attempt to rewrite any links it finds in a
|
|
/// single response. This is the method that underlies [`Server::set_autorewrite()`],
|
|
/// but by calling it on your responses manually, you can choose exactly what
|
|
/// responses are rewritten.
|
|
/// * [`Request::rewrite_path()`] will rewrite a single link. This method works best
|
|
/// for when you need a lot of precision, like including links that need to be
|
|
/// rewritten alongside links that don't, or when you're rewriting links in
|
|
/// responses that aren't `text/gemini`.
|
|
///
|
|
/// All of these methods work on both `scgi_srv` and `gemini_srv` modes, so you can
|
|
/// use them regardless of what you plan on compiling your server to, which is
|
|
/// recommended if you're planning to offer compilation to either, or if you would
|
|
/// like to be able to change later. It's worth noting that while it will *work*, on
|
|
/// `gemini_srv` mode, `gemini_srv` servers are *always* mounted at the base path, so
|
|
/// this method really won't do anything other than sign off that the link is good.
|
|
///
|
|
/// If there's a problem with rewriting the URLs, typically because the proxy
|
|
/// server/SCGI client being used doesn't correctly implement the SCGI spec, then any
|
|
/// `text/gemini` responses bearing an absolute link will be `40 TEMPORARY FAILURE`,
|
|
/// and an error will be logged explaining what went wrong.
|
|
///
|
|
/// For more information about how rewritten paths are calculated, see
|
|
/// [`Request::rewrite_path()`].\
|
|
/// For more information about what responses are rewritten,
|
|
/// see [`Response::rewrite_all()`].
|
|
pub fn set_autorewrite(mut self, autorewrite: bool) -> Self {
|
|
self.autorewrite = autorewrite;
|
|
self
|
|
}
|
|
|
|
fn build(mut self) -> Result<ServerInner> {
|
|
#[cfg(feature = "gemini_srv")]
|
|
let config = cert::tls_config(
|
|
&self.cert_path,
|
|
&self.key_path,
|
|
#[cfg(feature="certgen")]
|
|
self.certgen_mode
|
|
).context("Failed to create TLS config")?;
|
|
|
|
self.routes.shrink();
|
|
|
|
#[cfg(feature="user_management")]
|
|
let data_dir = self.data_dir;
|
|
|
|
Ok(ServerInner {
|
|
routes: Arc::new(self.routes),
|
|
timeout: self.timeout,
|
|
complex_timeout: self.complex_body_timeout_override,
|
|
autorewrite: self.autorewrite,
|
|
#[cfg(feature = "gemini_srv")]
|
|
tls_acceptor: TlsAcceptor::from(config),
|
|
#[cfg(feature="ratelimiting")]
|
|
rate_limits: Arc::new(self.rate_limits),
|
|
#[cfg(feature="user_management")]
|
|
manager: UserManager::new(
|
|
self.database.unwrap_or_else(move|| sled::open(data_dir).unwrap())
|
|
)?,
|
|
})
|
|
}
|
|
|
|
/// Start serving requests on a given bound address & port
|
|
///
|
|
/// `addr` can be anything `tokio` can parse, including just a string like
|
|
/// "localhost:1965"
|
|
pub async fn serve_ip(self, addr: impl ToSocketAddrs + Send) -> Result<()> {
|
|
let server = self.build()?;
|
|
let socket = TcpListener::bind(addr).await?;
|
|
server.serve_ip(socket).await
|
|
}
|
|
|
|
#[cfg(feature = "scgi_srv")]
|
|
/// Start serving requests on a given unix socket
|
|
///
|
|
/// Requires an address in the form of a path to bind to. This is only available when
|
|
/// in `scgi_srv` mode.
|
|
pub async fn serve_unix(self, addr: impl AsRef<std::path::Path>) -> Result<()> {
|
|
let server = self.build()?;
|
|
let socket = UnixListener::bind(addr)?;
|
|
server.serve_unix(socket).await
|
|
}
|
|
}
|
|
|
|
impl Default for Server {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
async fn send_response_header(response: &Response, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
|
|
|
|
let meta = if response.meta.len() > 1024 {
|
|
warn!("Attempted to send response with META exceeding maximum length, truncating");
|
|
&response.meta[..1024]
|
|
} else {
|
|
&response.meta[..]
|
|
};
|
|
|
|
let header = format!(
|
|
"{status} {meta}\r\n",
|
|
status = response.status,
|
|
meta = meta,
|
|
);
|
|
|
|
stream.write_all(header.as_bytes()).await?;
|
|
stream.flush().await?;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
async fn send_response_body(mut body: Option<Body>, stream: &mut (impl AsyncWrite + Unpin + Send)) -> Result<()> {
|
|
match &mut body {
|
|
Some(Body::Bytes(ref bytes)) => stream.write_all(bytes).await?,
|
|
Some(Body::Reader(ref mut reader)) => { io::copy(reader, stream).await?; },
|
|
None => {},
|
|
}
|
|
|
|
if body.is_some() {
|
|
stream.flush().await?;
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
#[cfg(feature="ratelimiting")]
|
|
/// Every 5 minutes, remove excess keys from all ratelimiters
|
|
async fn prune_ratelimit_log(rate_limits: Arc<RoutingNode<RateLimiter<IpAddr>>>) -> Never {
|
|
let mut interval = interval(tokio::time::Duration::from_secs(10));
|
|
let log = rate_limits.as_ref();
|
|
loop {
|
|
interval.tick().await;
|
|
log.iter().for_each(RateLimiter::trim_keys_verbose);
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "ratelimiting")]
|
|
enum Never {}
|
|
|
|
async fn opt_timeout<T>(duration: Option<time::Duration>, future: impl Future<Output = T>) -> Result<T, time::error::Elapsed> {
|
|
match duration {
|
|
Some(duration) => time::timeout(duration, future).await,
|
|
None => Ok(future.await),
|
|
}
|
|
}
|