Squish the Stack
We're not kidding about it being resource stingy. Here's the Rampart HTTP server running on my machine as I'm coding this web page on an Intel Mac with almost all of its modules loaded:
HTTP, HTTPS, and Websockets
Rampart's HTTP/HTTPS server is based on a modded libevhtp_ws and libevent2. Benchmarks showed that libevhtp outperformed Nginx (which we really respect), so that's why we chose it. We haven't benchmarked it against Node yet but it's reasonable to assume we'll keep pace while consuming a lot less of the host's resourcesmore info...
Full Text Search with SQL
We've licensed the same fulltext and relational database engine that powered eBay's auction search and 100's of other large sites across the internet. The SQL engine is especially good at serving database driven content. It is also realtime so that new or changed documents are instantly searchable. We've included a CSV parser so you can migrate data into it as well.more info...
We've included direct REDIS integration for all the obvious reasons one might want to use it. We also crafted a custom XREAD command to work like PUB/SUB so you can, in combination with WebSockets, create chats, channels and DMs that don't require the extra step of saving, retrieving and sorting messages from a message queue.more info...
Lightning Memory-mapped Database
LMDB is a fast, truly ACID compliant RAM based key-value store. It's read biased and serves binary or text data really well. We've added automatic conversion to/from CBOR and JSON to make it that much easier. It's not as fast as Redis when in full ACID mode, but then again it'll never lose data when Redis might. LMDB startup is relatively instant, even on large databases, and it's persistent to disk in case of unexpected interruption.more info...
Rampart gives you the power to crawl and scrape in a very flexible manner. However, with great web crawlers come great responsibilities. The ROBOTS.TXT module uses Google's robots.txt code so you can avoid abusing sites that do not wish to have portions of their content crawled.more info...
Using a modded version of HTMLTidy Rampart provides the ability to manipulate and traverse HTML in a manner somewhat akin to JQuery, but on the server side. This module can also clean up and format messy or compacted HTML after it's been CURL'ed from somewhere. Note: I used this module several times to locate errant imbalaced <div>s when writing this site.more info...
The OpenSSL library module provides the ability to safely encrypt and decrypt stuff generate crypto-hashes, generate RSA key pairs, and more. It also provides cryptographic quality random and Gaussian random number generation. Sorry, it doesn't generate crypto-currency though. Note: for speedy non-crypographic hashing see the non-cryptographic hashes.more info...
All the common commands and functions required to manipulate files and talk to the OS are in here (hopefully): printf(), sprintf(), fopen(), fread(), fwrite(), hexify(), dehexify(), trim(), stat(), exec(), shell(), kill(), mkdir(), rmdir(), sleep()... and a lot more.more info...
Fast HyperLogLog, non-crypto Hash, & Random
We implemented our own super-fast HyperLogLog algorithm for on-line counting of distinct things. e.g. unique IP addresses. It's substantially faster and more accurate than Redis's. Our HLL uses Google's City Hash algorithm internally, but we also expose City Hash and its cousin Murmur Hash, as well as a fast pseudo random number generator based on XorShift64.more info...
More to come
We've been working on assembling Rampart for alpha release for about two Covid impaired years now. Some stuff hasn't made it into it yet, but we are continuing to add features and by no means are we nearly done adding things. There's another really fast memory-mapped DB that was supposed to be in here (but I couldn't make the deadline). It should exceed Redis' and LMDB's speed while being a lot more memory efficient. Keep an eye out for more news.
Why did we do this?
While NodeJS is extremely popular and NPM provides a ton of add-ons quickly, on the whole it's pretty fat. By the time you configure MariaDB and Apache Lucene you've eaten a day (or more) and a considerable amount of resources on the host machine. Additionally, it's not that much fun.
In creating Rampart we sought to create a "full stack" that was simple and fast to download, configure, and start creating whatever application you had in mind in a time frame measured in minutes rather than days. User machines, phones, and IOT devices need to serve their primary role without impediment. Rampart's job is to support that role and not get in the way. OTOH if you dedicate a machine or cluster to a Rampart task you'll get a lot more done with a lot less investment.
Who are we?
Moat Crossing Systems LLC, the makers of Rampart, is a couple of guys who between them have developed software and applications that have served many billions of pages of database backed content on the internet. We had some summer help from a couple of Columbia students who helped up keep tabs on the "current-think" and coded some of the OS Utility functions. It all started in January of 2020 with a phone conversation where we were bitching about how un-fun web development had become, and how resource heavy everything seemed to be. A comment was made that if we could full-text index Wikipedia and serve it up on a Raspberry Pi Zero, then we'd have something unique that people might want. So that's what we did.
We hope you will give this thing a try. Contact us here or at if you have questions or need help with something. Maybe later on we'll get more sophisticated with our support infrastructure, but for now one of us will try to get back to you as soon as possible.
Here's a photo of our fancy server farm before it was installed in a rack at the ISP. There's four Pi Zeros and two Pi 4s. The four Zeros are mirror copies of one another just in case *stuff* happens. The two Pi 4's allow us to do development and builds a bit faster than the Zeros allow us. The "bush" is proxied by an NginX server running on a fairly large old Xeon machine. This made certificate managment and other things somewhat easier on us.
Please note that we have no particular bias towards producing software principally for the Raspberry Pi. The whole purpose for its usage here is to demostrate how much Rampart can do with minimimal resources. That way you can extrapolate how fast and efficient it will be on larger Intel, AMD, and ARM platforms.
|>> |>> __|__ __|__ \ | / \ / | ^ | | ^ | __| o |__________| o |__ [__|_|__|(rp)| | |______] ____[|||||||||||||__|||||||||]____