André Carlucci

Skeptic .net development

You don’t always need a database

As we begin a new project, we quickly start thinking about what kind of database we are going to use. Options like Oracle, SqlServer, Mysql or even a NoSql database like Mongo or Cassandra always pop up like a good choice.

However, such databases each occupy tens or hundreds of megabytes (gigas?) on disk, besides the fact of another installation, deploy, support, more costs a lot of our time.

Is it worth it?

For most of the time our applications will be storing just a small amount of data. Even thousands of users, clients and products would be taking just some megabytes of your disk. Is it worth it to install a whole database (on another server) just for that? And what if you still don’t know the schema you will end up using and you still have to change the tables all the time?

I’ve been thinking about that for quite some time and one day I found this article by Rob Conery talking about lists being persisted by a json store, with all linq goodness, speed and simplicity. I went crazy about it and started using the library right away in a small project of mine.

I really liked the framework, but very quickly a lot of people jumped in and the library lost its simplicity, stability and you could never know which repository or nuget were the official one. It’s not even on Github anymore :(

Biggy became too big, so I created Mini-Biggy

I started from scratch using the same concept, but adding a lot of features and taking care for not adding complexity. The goal was to add a Nuget package to your project and start coding. Simple, awesome, fast!

So how it works?

Create your project and install MiniBiggy using Nuget:

Install-Package mini-biggy

Create a class called Tweet.

The code above will create a file called tweets.data with our tweet list serialized. Call Save() or SaveAsync() to persist the list.

And you are all set.

Loading them later

Every time you create a list of some type, it will load all saved objects of that type:

Linq Goodness

Remember, your list is entirely in memory, so any linq operator will work just fine:

And it’s so fast! Find something in a list of 1,000,000 takes only some milliseconds. In-memory lists are awesome!

Serializers

Mini-biggy comes with 2 serializers:

  • JsonSerializer
  • PrettyJsonSerializer (the json is indented)

You can change that behavior every time you load your list.

Saving Modes

You can follow 3 strategies to save your list using mini-biggy:

Manual Save

The list will be saved only when Save() or SaveAsync() is called. This is the default behavior.

Automatic Save

The list will be saved on every change: adding, deleting or updating an item saves the list. You still can call Save and SaveAsync if you want.

Just note that it can take some time to save, specially on loops (use AddRange when on loops).

Background Save

This is really useful if your list changes a lot, specilly by multithread applications (web). The list will be saved every X seconds, but only if it was modified. You still can call Save and SaveAsync if you want.

Web Applications

Web applications are multithread applications by nature and there will be a lot of requests trying to access our persistent list at the same time. Don’t worry, the list is thread-safe. Just make sure to register it as a singleton on your DI of choice, use the Background Save strategy and you’re all set.

Performance

I did some non-scientific-quick-and-dirty performance tests just to have an idea about how big MiniBiggy can get.

Here is the result:

Number of Items Save Load Find <3 File on Disk Memory
1,000 0.120s 0.005s 0.001s 61kB 0.09MB
1,000,000 0.350s 0.335s 0.005s 6.54MB 9.91MB
10,000,000 3.248s 3.985s 0.039s 68.345MB 99.17MB

As you can see, things are really fast until you reach some millions of registers.

Is mini-biggy for you?

Mini-biggy is an excellent choice for storing your persistence data if:

  • you want a freaking fast and simple way to store your objects.
  • you have less than a couple million objects (beware for mobile devices)
  • you want full Linq support
  • your objects are json serialized and you will take care of them when changing your model

Where is the source code? How can I help?

Find more about the project on https://github.com/mini-biggy/mini-biggy!