FlockDB

Last updated
FlockDB
Original author(s) Nick Kallen, Robey Pointer, John Kalucki and Ed Ceaser from Twitter
Developer(s) Twitter [1]
Initial releaseApril 2010 (April 2010)
Final release
1.8.5 / 23 February 2012;11 years ago (2012-02-23) [2]
Repository
Written in Scala, Java, Ruby
Type Graph Database
License Apache License 2.0
Website github.com/twitter/flockdb

FlockDB was an open-source distributed, fault-tolerant graph database for managing wide but shallow network graphs. [3] It was initially used by Twitter to store relationships between users, e.g. followings and favorites. FlockDB differs from other graph databases, e.g. Neo4j in that it was not designed for multi-hop graph traversal but rather for rapid set operations, not unlike the primary use-case for Redis sets. [4] FlockDB was posted on GitHub shortly after Twitter released its Gizzard framework, which it used to query the FlockDB distributed datastore. The database is licensed under the Apache License. [1]

Contents

Twitter no longer supports FlockDB. [5]

See also

Related Research Articles

<span class="mw-page-title-main">Mantis Bug Tracker</span> Bug tracking system

Mantis Bug Tracker is a free and open source, web-based bug tracking system. The most common use of MantisBT is to track software defects. However, MantisBT is often configured by users to serve as a more generic issue tracking system and project management tool.

HBase is an open-source non-relational distributed database modeled after Google's Bigtable and written in Java. It is developed as part of Apache Software Foundation's Apache Hadoop project and runs on top of HDFS or Alluxio, providing Bigtable-like capabilities for Hadoop. That is, it provides a fault-tolerant way of storing large quantities of sparse data.

Moose File System (MooseFS) is an open-source, POSIX-compliant distributed file system developed by Core Technology. MooseFS aims to be fault-tolerant, highly available, highly performing, scalable general-purpose network distributed file system for data centers. Initially proprietary software, it was released to the public as open source on May 30, 2008.

<span class="mw-page-title-main">Redis</span> Open-source in-memory key–value database

Redis is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, HyperLogLogs, bitmaps, streams, and spatial indices. The project was developed and maintained by Salvatore Sanfilippo, starting in 2009. From 2015 until 2020, he led a project core team sponsored by Redis Labs. Salvatore Sanfilippo left Redis as the maintainer in 2020. It is open-source software released under a BSD 3-clause license. In 2021, not long after the original author and main maintainer left, Redis Labs dropped the Labs from its name and now is known simply as "Redis".

Gizzard was an open source sharding framework to create custom fault-tolerant, distributed databases. It was initially used by Twitter and emerged from a wide variety of data storage problems. Gizzard operated as a middleware networking service that ran on the Java Virtual Machine. It managed partitioning data across arbitrary backend datastores, which allowed it to be accessed efficiently. The partitioning rules were stored in a forwarding table that maps key ranges to partitions. Each partition managed its own replication through a declarative replication tree. Gizzard handled both physical and logical shards. Physical shards point to a physical database backend whereas logical shards are trees of other shards. In addition Gizzard also supported migrations and gracefully handled failures. The system was made eventually consistent by requiring that all write operations are idempotent and commutative. As operations fail they are retried at a later time. Gizzard is available at GitHub and licensed under the Apache License 2.0.

Riak is a distributed NoSQL Key-value data store that offers high availability, fault tolerance, operational simplicity, and scalability. Riak moved to an entirely open-source project in August 2017, with many of the licensed Enterprise Edition features being incorporated. Riak implements the principles from Amazon's Dynamo paper with heavy influence from the CAP Theorem. Written in Erlang, Riak has fault tolerant data replication and automatic data distribution across the cluster for performance and resilience.

<span class="mw-page-title-main">OrientDB</span>

OrientDB is an open source NoSQL database management system written in Java. It is a Multi-model database, supporting graph, document, key/value, and object models, but the relationships are managed as in graph databases with direct connections between records. It supports schema-less, schema-full and schema-mixed modes. It has a strong security profiling system based on users and roles and supports querying with Gremlin along with SQL extended for graph traversal. OrientDB uses several indexing mechanisms based on B-tree and Extendible hashing, the last one is known as "hash index", there are plans to implement LSM-tree and Fractal tree index based indexes. Each record has Surrogate key which indicates position of record inside of Array list, links between records are stored either as single value of record's position stored inside of referrer or as B-tree of record positions which allows fast traversal of one-to-many relationships and fast addition/removal of new links. OrientDB is the fifth most popular graph database according to the DB-Engines graph database ranking, as of December 2021.

<span class="mw-page-title-main">ERPNext</span> Enterprise Resource Planning software

ERPNext is a free and open-source integrated Enterprise Resource Planning (ERP) software developed by an Indian software company Frappe Technologies Pvt. Ltd. and is built on MariaDB database system using Frappe, a Python based server-side framework.

<span class="mw-page-title-main">Fat-Free Framework</span>

Fat-Free Framework is an open-source web framework distributed under the GNU General Public License and hosted by GitHub and SourceForge. The software seeks to combine a full featureset with a lightweight code base while being easy to learn, use and extend.

Meteor, or MeteorJS, is a partly proprietary, mostly free and open-source isomorphic JavaScript web framework written using Node.js. Meteor allows for rapid prototyping and produces cross-platform code. The server-side MongoDB program is the only proprietary component of Meteor and is part of the Meteor download bundle. It is possible to use Meteor without using the server-side MongoDB. It uses the Distributed Data Protocol and a publish–subscribe pattern to automatically propagate data changes to clients without requiring the developer to write any synchronization code. On the client, Meteor can be used with any popular front-end JS framework, Vue, React, Svelte, Angular, or Blaze.

<span class="mw-page-title-main">Apache Spark</span> Open-source data analytics cluster computing framework

Apache Spark is an open-source unified analytics engine for large-scale data processing. Spark provides an interface for programming clusters with implicit data parallelism and fault tolerance. Originally developed at the University of California, Berkeley's AMPLab, the Spark codebase was later donated to the Apache Software Foundation, which has maintained it since.

<span class="mw-page-title-main">Deeplearning4j</span> Open-source deep learning library

Eclipse Deeplearning4j is a programming library written in Java for the Java virtual machine (JVM). It is a framework with wide support for deep learning algorithms. Deeplearning4j includes implementations of the restricted Boltzmann machine, deep belief net, deep autoencoder, stacked denoising autoencoder and recursive neural tensor network, word2vec, doc2vec, and GloVe. These algorithms all include distributed parallel versions that integrate with Apache Hadoop and Spark.

RethinkDB is a free and open-source, distributed document-oriented database originally created by the company of the same name. The database stores JSON documents with dynamic schemas, and is designed to facilitate pushing real-time updates for query results to applications. Initially seed funded by Y Combinator in June 2009, the company announced in October 2016 that it had been unable to build a sustainable business and its products would in future be entirely open-sourced without commercial support.

<span class="mw-page-title-main">RocksDB</span>

RocksDB is a high performance embedded database for key-value data. It is a fork of Google's LevelDB optimized to exploit many CPU cores, and make efficient use of fast storage, such as solid-state drives (SSD), for input/output (I/O) bound workloads. It is based on a log-structured merge-tree data structure. It is written in C++ and provides official language bindings for C++, C, and Java; alongside many third-party language bindings. RocksDB is open-source software, and was originally released under a BSD 3-clause license. However, in July 2017 the project was migrated to a dual license of both Apache 2.0 and GPLv2 license, possibly in response to the Apache Software Foundation's blacklist of the previous BSD+Patents license clause.

<span class="mw-page-title-main">Apache Apex</span>

Apache Apex is a YARN-native platform that unifies stream and batch processing. It processes big data-in-motion in a way that is scalable, performant, fault-tolerant, stateful, secure, distributed, and easily operable.

<span class="mw-page-title-main">Dat (software)</span> Data distribution tool

Dat is a data distribution tool with a version control feature for tracking changes and publishing data sets. It is primarily used for data-driven science, but it can be used to keep track of changes in any data set. As a distributed revision control system it is aimed at speed, simplicity, security, and support for distributed, non-linear workflows.

<span class="mw-page-title-main">Pop PHP Framework</span> Free and open source PHP web framework

The Pop PHP Framework a free and open source PHP Web framework that was created by Nick Sagona. It is distributed under the BSD License and hosted on GitHub. The framework is intended to be utilized for rapid application development, with an emphasis on web applications.

<span class="mw-page-title-main">Enonic XP</span>

Enonic XP is a free and open-source web application platform and content management system (CMS) in one based on Java and Elasticsearch. Developed by the Norwegian software company Enonic, the microservice web platform can be used to build progressive web applications, Next.js websites, or web-based APIs. Enonic XP uses an application framework for coding server logic with JavaScript, and has no need for SQL as it ships with an integrated content repository. The CMS is fully decoupled, meaning developers can create traditional websites and landing pages, or use XP in headless mode, that is without the presentation layer, for loading editorial content onto any device or client. Enonic is used by major organizations in Norway, including the national postal service Norway Post, the insurance company Gjensidige, the national lottery Norsk Tipping, the Norwegian Labour and Welfare Administration, and all the top football clubs in the national football league for men, Eliteserien.

TerminusDB is an open source knowledge graph and document store. It is used to build versioned data products. It is a native revision control database that is architecturally similar to Git. It is listed on DB-Engines.

Originally developed in 2019 by software giant, Microsoft. under the name Coco, and later rebranded to Confidential Consortium Framework; (CCF) is an open-source framework for the development of a new category performant applications that focus on the optimization of secure multi-party computation and data availability. Intended to accelerate the adoption of blockchain technology by enterprise; CCF can enable a variety of high-scale, confidential permissioned distributed ledger networks that meet key enterprise requirements.

References

  1. 1 2 "FlockDB System Properties". db-engines.com. Retrieved 2017-10-09.
  2. "FlockDB 1.8.5 released". Twitter. February 22, 2012. Retrieved 2 November 2013.
  3. "Gigaom | Twitter Open-sources the Home of Its Social Graph". gigaom.com. Retrieved 16 September 2017.
  4. "GitHub - twitter-archive/flockdb: A distributed, fault-tolerant graph database". github.com. Retrieved 16 September 2017.
  5. "GitHub - twitter-archive/flockdb: A distributed, fault-tolerant graph database". github.com. Retrieved 27 May 2020. Twitter is no longer maintaining this project or responding to issues or PRs.