Beta released. If you like Dataplane, give it a star ⭐️ on Github.
Data Platform. Data Mesh.
Data Engineered. Dataplane.
Engineer the next generation of data platforms with Dataplane. The data platform that quickly gets you to reliable, robust and timeous data flowing throughout your data ecosystem.
Your data ecoystem connected.
Snap snap.Data pipelines in a flash.
Speed up iteration, fail fast and get to solid data pipelines at record speed.
Code. Test. Run.
Develop, test and run code directly in Dataplane's code editor. Dataplane's online code editor improves collaboration, provides shared compute resources and removes the need to buy, install and configure developer spec laptops. A faster way to get your data team up and running.
Environments. Data Domains. Data Mesh.
With Dataplane, you can now configure isolated environments.
These secure environments provide your data team with the following capabilities:
- Route to live - develop, test and deploy from development to production.
- Data domains with restricted access in your data mesh.
- Spin up test and learn projects for with colleagues outside the data team.
- Create safe playpens to discover and analyse live data.
Distributed computing.
Dataplane has unique load balancing functionality to scale, monitor and distribute analytical workloads.
Cloud native by design, Dataplane scales horizontally with your data.
You can even specify a specific group of workers to process a part of a data pipeline.
Super secret.
Data teams constantly require secure access to data resources such as databases or cloud storage.
With Dataplane, you can give your team access to secrets without revealing the actual secret.
Secrets remain secret by redacting the confidential information from the logs.
Secrets can be stored in Dataplane using a high level of encryption, retrieved from your Key Management Service (KMS) or provided to Dataplane through environment variables.
Cloud native. Works with your chosen cloud provider.
Collaborate as a team
Build on each other’s strengths by inviting your team members to co-create, manage and deploy data pipelines.
Give your team members the freedom to safely operate through fine-grained roles, responsibilities and permissions.
Schedule across timezones.
Dataplane makes it easier to set recurring schedules for your workflows.
There are two types, CRON which provides up to per minute scheduling and per second scheduling. You can also set different schedules across the globe by changing the timezone per schedule.
There are template examples for most CRON cases. You can adapt these examples to quickly set your CRON schedules.
Deployments.
With Dataplane deployments, you can now deploy to production with confidence.
Deployments package up your data pipelines into a version controlled isolated run time that cannot be edited. This allows you to continue to develop your pipelines without interrupting your mission-critical operations.
Speed. Scale. Resilience. Security. Lightweight.
Written in Golang and compiled to machine code, Dataplane offers exceptional performance for your data platform. The front end is built in React and interacts with the back end through secured JWT tokens, GraphQL and Websockets.
A Dataplane cluster can horizontally scale to many instances connected over a NATS message bus, providing self-healing, resilience and scale. Dataplane reduces your cloud expenditure by being super lightweight, only requiring minimal CPU and memory per instance. Dataplane also leverages advanced techniques such as leader election and distributed locking to avoid scheduling conflicts while adding on more instances.
