DC/OS: Meet the Data Center Operating System
The Data Center Operating System is a cluster “operating system” based on Apache Mesos that provides an unified framework to run, manage and log operations against the cluster as if it was just one single, huge mainframe. Let’s look a bit at what DC/OS is and what it can do for you.
What is DC/OS?
DC/OS is a distributed operating system based on the Apache Mesos distributed systems kernel. It enables the management of multiple machines as if they were a single computer. It automates resource management, schedules process placement, facilitates inter-process communication, and simplifies the installation and management of distributed services. Its included web interface and available command-line interface (CLI) facilitate remote management and monitoring of the cluster and its services.
To put it simply, imagine what your favorite operating system (e.g. Linux) does on your computer, and imagine an operating system capable of doing the same thing across thousands of machines. There you have DC/OS. The Data Center Operating System abstracts all the resources in the data center and makes them look like one huge computer. Every program talking to the operating system can request RAM, CPU and network, the advantage of using a data-center scale operating system is that the program can request those very same resources from a larger pool of resources rather than your laptop only.
DC/OS started as a commercial software developed by the San Francisco-based Mesosphere. Somewhere in 2015 there were rumors about Microsoft trying to buy Mesosphere but nothing was publicly disclosed. Soon after, Microsoft added DC/OS to its Azure products. In 2016, since release 1.16, DC/OS was open sourced and made available for free. This helped the software gain traction as Mesosphere thought its product wouldn’t be as successful as other emerging products such as Kubernetes at the time. Shortly after the open sourcing of the project, Autodesk announced the software helped reduce its AWS instances by 66%.
DC/OS: Sleek tools for tomorrow’s datacenter
DC/OS Web interface
DC/OS comes with three interfaces: web UI, CLI and API. Although the API is the most important to developers and for integration purposes let’s stop to take a look at one of the most neglected: the web interface. The web UI is a polished, modern interface that provides all the useful information about your cluster at a glance. The amount of information displayed with a clean look and structure is astonishing.
As you can see from the above image all the information, with a focus on statistics, is available one click away and the left menu is a showcase of how easy it is to get started with DC/OS when it’s up and running. One of the greatest features that you can get a glance from the image above is the package manager (more in a while…)
The command line interface is a useful utility when performing tasks in a timely manner and when your eyes are not glued to the gorgeous web interface.
Apache Mesos: The Data Center Operating System needs a Data Center Kernel
All the capabilities boasted by DC/OS can be traced back to one single component: its kernel. DC/OS is based on Apache Mesos, a distributed cluster manager that provides common functions expected in enterprise environment such as:
- Scalability
- High availability
- Isolation
- Scheduling
All these complex functions are implemented out of the box by the Mesos kernel that handles all the request in a highly available, scalable fashion, still providing isolation among components and scheduling workloads selecting the best hosts to run the task on.
However powerful Mesos is there are things that aren’t available out of the box such as name resolution (DNS) or package management, and that’s where DC/OS plugs in.
Running a Marathon has never been so easy!
Marathon is a Mesos framework (read more in this article) that acts like an “init system” (much like systemD) that operates at cluster level. Thanks to Marathon you can easily control what is running on your cluster and promptly stop or restart workloads.
Marathon is a key component in a DC/OS cluster since it allows complex applications such as Kubernetes to run on top of the cluster with minimal effort. As a matter of fact Marathon is an underlying orchestration services that is used to manage Mesos containers.
Chronos: take control of your task long schedule
Marathon enables you to control what is running on your cluster, but it assumes these services will run just like programs do and only manual intervention/failures can modify this flow. Marathon isn’t really suited for long tasks such as Hadoop ones, and this is where Chronos steps in.
Chronos is a cron-like Mesos framework that allows the administrator to define tasks to run against the cluster in a cron-fashion. As a matter of fact the syntax closely resembles cron. With marathon taking care of components and Chronos taking care of time schedules the Data Center Operating System is getting close to a real operating system.
Control packages in the Universe
One of the most useful features of Linux-based operating systems is the package management. DC/OS has its own package manager (called Cosmos) that enables the administrator to install complex software such as Hadoop or Cassandra in a single click or as simply as a command line command: $ dcos package install PACKAGE .
Just like package managers, Cosmos needs repositories. The main repository for DC/OS is Mesosphere Universe. Universe includes many battle-tested software such as Hadoop and Cassandra ready to be installed in your DC/OS cluster.
Of course the Universe repository is just metadata and in the future new third-party repositories may pop up.
DC/OS or Kubernetes?
One of the recurring questions about DC/OS is whether to adopt it or Kubernetes. Kubernetes and the Data Center Operating System mostly cover the same niche, however the latter is better for Big Data environments such as Hadoop since it bridges container technologies with more traditional ones. Kubernetes can only handle containers while the latter aims to orchestrate any program thrown at it, you can even run Kubernetes on top of DC/OS with a single-click install.
Kubernetes isn’t going anywhere soon, however if you want to keep the door open to other technologies in the future (and you’re willing to carry the burden) go with DC/OS.
DC/OS or OpenStack?
Another recurring comparison is the one between DC/OS and OpenStack. OpenStack is a IaaS solution whereas DC/OS is more a container platform with great builtin capabilities. In the period between 2013 and 2015 the two projects were directly compared by many people but they never truly went into competition against each other.
Nowadays they both fill their own niche, and OpenStack and DC/OS mixed environments aren’t exactly common. Nevertheless since the OpenStack Ocata release, OpenStack can handle DC/OS through Magnum.
- 2020 A year in review for Marksei.com - 30 December 2020
- Red Hat pulls the kill switch on CentOS - 16 December 2020
- OpenZFS 2.0 released: unified ZFS for Linux and BSD - 9 December 2020
Recent Comments