Skip to main content
All CollectionsDeveloper & Technical Docs
So, how does Hivenet's cloud storage work?
So, how does Hivenet's cloud storage work?

How does Hivenet work? Learn about Hivenet's distributed cloud storage, key architecture concepts, and how data is stored and retrieved.

Daniel Awbery avatar
Written by Daniel Awbery
Updated over a week ago

If you’re reading this, we guess you already have some idea of what Hivenet is and how it works. There’s even a page on our site for that. However, if you’re anything like us, more knowledge is never a bad thing (unless you spoil the ending of films… don’t do that… ヽ(ಠ_ಠ)ノ).

With this in mind, we want to share more details about Hivenet’s architecture and some key concepts that make our services possible. This knowledge isn’t just “nice to have”—in 2025, Hivenet will move to a new version of its architecture, with foundational changes to our implementation.

More details will come later in 2025, but for now, let’s dig in …

Start at the beginning: what does Hivenet do?

Before we talk about how Hivenet works, we think it’s best to start by understanding a few key mechanics and what solution it provides for our users. Please note that this article will focus on Hivenet’s cloud storage service to reduce complexity.

Imagine you want to store a file on Hivenet - from the user’s perspective, it seems as easy as compress file, upload file, polite applause … but there’s a lot more happening behind the scenes. Figure 1 below approximates the lifecycle of a user’s files, and from this, we can tease out two of the core challenges our service addresses:

  1. How do I take 1 .. n files and distribute them across a network of storage nodes?

  2. How do I retrieve my files from this network?

Figure 1: the high-level lifecycle of files on Hivenet

We know there’s probably at least one person among our readers right now raising their hand to say: “Aren’t these the types of challenges that IPFS exists for”? And yes, we could short-circuit this discussion by saying that Hivenet leverages the InterPlanetary File System (“IPFS”) model to implement a distributed peer-to-peer network.

But if you don’t know what IPFS is, or if you’re the kind of person who prefers to take their time to enjoy a game rather than speedrun it, then why don’t we do just that: Why don’t we slow down, close our eyes, and picture a peaceful orchard in rolling hills … and the gentle sound of bees.

Hivenet: the clue is in the name

In the previous section, we used Figure 1 to sketch the lifecycle of a file on Hivenet. To start unpacking what underpins this process, let’s turn to Figure 2 below, which shows the highest-level building blocks and roles in our architecture. You may notice, right at the center of it all, the ”Hivenet Swarm” - but what is a swarm in this context?

In the IPFS model, a swarm is described as:

The network of IPFS peers with which your local node has connections. Swarm addresses are addresses that your local node will listen on for connections from other IPFS peers.

Too technical at this stage? We agree! So let’s instead turn to nature, where many things come in swarms: birds, jellyfish, terrifying nano-robots … We at Hivenet prefer to think of swarms of bees: the hard-working, community-minded honey-makers. If you know a little about bee colonies, they are intricate, self-governing societies, and you may also think that this has something to tell us about how Hivenet functions.

Figure 2: a (very) high-level view of Hivenet’s architecture

Figure 3 below shows the three categories of adult honeybees. You can find a lot more detail here, but for our purpose, we will just note their key characteristics:

  • The Queen: one per colony; lays eggs and organizes the hive’s work through pheromone signals; lifespan of ~3-4 years.

  • Drones: ~100s per colony; born from unfertilized eggs to live a short life; their only role is to mate with queens from different colonies.

  • Workers: ~10 per drone; born from fertilized eggs for a longer lifespan; care for the queen, maintain the hive, gather resources.

As you can see, regardless of size, number, or lifespan, the hive will not function without enough of each category—every bee has a role to play in sustaining the collective.

Figure 3: the three types of adult honeybees [source]

Table 1 maps the functions of a honeybee colony to components in Hivenet’s architecture. These mappings are not precise, though we hope this comparison helps bring each part of Hivenet’s architecture to life, along with their role in achieving the aims of the collective.

Honeybees

Function

Hivenet

Function

Beekeeper

Tends a farm of bee hives; responsible for the well-being and safety of the bees

Hivenet’s Team

Hivenet’s talented people keep the network safe and operational

Beehive

Tangible structure that houses a honeybee colony; a honeycomb with different cells for bees, eggs, honey, etc.

User Workspace

A per-user data structure that encapsulates all data and data volumes associated with the user

Colony

A single bee family; consists of one queen, many drones, and thousands of workers

User-associated Resources

All resources associated with the user workspace, including their apps, storage volumes, and contributions

Queen

One per colony; lays eggs and organises the hive’s work through pheromone signals; lifespan of ~3-4 years.

Hivenet’s Mutable Platform

Provides relational structure to the colony; helps each bee know their place and role

Drones

~100s per colony; born from unfertilized eggs to live a short life; their only role is to mate with queens from different colonies.

Storage Nodes

Devices that participate in the network; host data from many users, disconnecting and reconnecting at any time

Workers

~10 per drone; born from fertilized eggs for a longer lifespan; care for the queen, maintain the hive, gather resources.

Hivenet’s Agents

Services that run on nodes across the network; handle crucial tasks to store and retrieve data in a secure, efficient way

Honey

A precious substance, refined from plant pollen; gathered by workers and stored in the hive for safekeeping

User Data

Our users’ precious data, kept safe through the collaboration between beekeeper and their bee colonies

Table 1: approximate mapping between Hivenet and the roles in a honeybee hive

Before moving on, we want to highlight one key theme of our analogy: a hive that loses its queen (a “queenless” hive) doesn’t immediately collapse. Instead, it tries to sustain itself but loses focus. The bees become more nervous and aggressive. The workers begin to prioritize foraging outside over tending the hive. Eventually, if no replacement is found, the hive will die off. Likewise, while Hivenet offers a distributed, peer-to-peer network, the individual nodes cannot function effectively without the support of the “Mutable Platform”.

We will revisit the Mutable Platform and its purpose later on, but first we need to reflect on the nature of change.

Heraclitus was right: only mutability is immutable

Heraclitus famously said that: “The only constant in life is change...”

… except that’s a misrepresentation based on how others expressed his ideas, though it’s still an important sentiment for our discussion. In our introduction, we described Hivenet’s goal as storing and retrieving many files using a distributed network of nodes. There’s just one small problem: time. Or rather the passage of time:

files are created and deleted;

folders are moved and renamed;

devices connect and disconnect …

This sounds like chaos! So what do we do?

Figure 4 below demonstrates how Hivenet models our users with a unique “workspace” containing its associated storage “volumes,” “folders,” and “files.” Let’s use this framing to describe how Hivenet tracks change over time (note: we will touch on the “chunks” that make up files in more detail in an upcoming article).

Figure 4: the Hivenet user and their associated data

Another famous quote attributed to Heraclitus is: “No one steps in the same river twice, for it is not the same river and they are not the same person”.

Along similar lines, if you store your resume on Hivenet and you edit it … is it the same file? This depends on your point of view:

  • User: “This is my resume, but … improved, maybe?”

  • System: “The number and value of bytes in this document differ. Therefore, this is a different file.”

To ensure a consistent user experience while recognizing that the underlying data changes, we must support both perspectives. For this, we need to introduce identifiers (“IDs”), the labels used to refer to objects in a system, specifically the “mutable identifier” and the “content identifier.” The IPFS site has a great overview of “Immutability,” but in simple terms, if something can be changed, it is “mutable” - and if it cannot, then it is “immutable.”

Table 2 explains how Hivenet applies these concepts to maintain mutable and immutable references at all layers of its architecture.

The mutable ID (mid)

A unique ID (specifically a uuid) that provides a stable reference for an object whose content can change.

The content ID (cid)

A unique ID (specifically a sha hash) that both refers to and derives from the content it identifies.

(Note: hashing is a complex topic, and we won’t go into more detail here).

The mutable record (mid/cid pair)

A combined identifier that points to the current content for a specific mid.

Table 2: key identifiers used in Hivenet

To illustrate how objects in Hivenet use mids and cids to identify each other, let’s take a look at Figure 5, which applies these concepts to our resume example as follows:

  1. September 19: resume updated; total words reduced by eight; file saved.

  2. File cid re-generated as 00004187000215

  3. File mid updated to reference 00004187000215

Figure 5: worked example of mids and cids; bold text represents changed data

At this point, you might say: “using two IDs to refer to one thing sounds like using two names to talk to a person … let’s get rid of mids!”. That is a possible approach but one that has implications - for example, see Table 3:

Updating a file with …

mids and cids

single change

Workspace mid

↓ References

Volume mid

↓ References

Folder mid

↓ References

File mid

↓ References

NEW: File CID

cids only

many changes

NEW: File cid

↓ Requires

NEW: Folder cid

↓ Requires

NEW: Volume cid

↓ Requires

NEW: Workspace cid

Table 3: comparing the impact of file changes with and without the mid

We can see that without mids, if we update the cid for a file, we must also update:

  • The folder’s cid that points to the file, then

  • The volume’s cid that points to the folder, then

  • The workspace’s cid that points to the volume, etc.

Extending this across many changes at once, we can anticipate the potential for a negative impact on performance. In contrast, with a stable mid, the impact of any change is restricted.

We covered a lot in this section: we demonstrated mids and cids through the example of a user updating their resume and laid out the hierarchy of objects (see Figure 4) that represents a user’s stored data. In the next two sections, we will briefly introduce how Hivenet works with these identifiers across the network in real-time - this deserves its own deep-dive, and we’ll provide one in a future article on upload/ download workflow.

New cids on the Block(s)

Firstly, apologies for the section title; it’s not the 90s … also, the last section was long, and you didn’t deserve that. So, what’s next? You may recall from Table 2 the “mutable record” that pairs an object’s mid with a cid identifying its latest version. But where do we find them?

Let’s take the example of a user uploading a folder of their files. During the upload process, a set of mutable records is created in Hivenet’s Mutable Platform, which you may also remember as the “Queen” of Hivenet’s colony. The Mutable Platform is a database service that allows nodes across the network to look up mutable records and identify the “blocks” of data associated with them. Working together, the Mutable Platform and the mutable records it contains are what allow us to:

  1. Locate where in the network a specific piece of data sits

  2. Identify and locate related pieces of data

  3. Download and recombine these pieces of data into your files on-demand

In the last two sections, we have gone into detail on how Hivenet uses identifiers to keep track of data blocks across its network. At this point, you may think, “Keeping track of everything sounds hard work … Who does all that work”?

Let’s find out.

Working 9 to 5 (except it’s bees, and they work 24/7)

Earlier (see Table 2), we discussed how worker bees manage everything around the hive (cleaning, foraging, making honey) and compared them to Hivenet’s “agents.” In the same way that worker bees perform different functions depending on their age and stage of development, Hivenet has agents tailored to specific tasks. For example, hivenet-http-agent processes request to store and retrieve data blocks from nodes on the network; hivenet-resource-agent provides APIs (application programming interface) to store and retrieve data blocks from a node’s local storage, and hivenet-relay-agent provides various network services to help the swarm of nodes communicate.

Figure 6 below expands on our earlier diagrams to give an impression of how agent-driven communication works for our resume example.

Figure 6: simplified communication flow for one file

We mentioned Hivenet’s transition to a new architecture in 2025 in the introduction. One of the key areas of improvement was to break one agent that handled many tasks into multiple specialized services. More details on how and why this change was made are discussed in the v2 migration overview.

Let’s now move to our conclusion, where we will recap the purpose of this article and summarise what we have learned.

So … what does Hivenet do again?

Our intended goal for this article was for you to understand how Hivenet works (focusing on our cloud storage service). We began our journey by asking what Hivenet does and gave an initial answer: we take users’ files, distribute them across a network, and retrieve the pieces on demand. And that’s a reasonable summary.

However, after comparing Hivenet to a colony of honeybees and misquoting a philosopher to reflect on handling change, you are now armed with much more detail on how these pieces come together to form a functioning, self-governing system.

So, let’s restate our earlier definition of “what Hivenet does” in more technical terms, confident that nothing here should scare you - the purpose of Hivenet is to provide:

  1. A data store for blocks, where each block is identified by a hash of its content.

  2. A database of mutable records, where each record maps a stable, long-term identifier to its associated blocks on a network.

Did this answer your question?