Insights and Updates

Read the HarperDB teams’ thoughts on database technology topics such as SQL, NOSQL, Edge Computing, IoT, Industry 4.0, and more

What is Persistent ETL?

May 08, 2019
If you’ve made it to this blog you’ve probably heard the term “persistent” thrown around with ETL , and are curious about what they really mean together. Extract, Transform, Load (ETL) is the generic concept of taking data from one or more systems and placing it in another system, often in a different format. Persistence is just a fancy word for storing data. Simply put, persistent ETL is adding a storage mechanism to an ETL process. That pretty much covers th e what, but the why is much more interesting…
Read More

What is Polyglot Persistence and Why is it awful?

April 23, 2019

According to Wikipedia,Polyglot persistence is the concept of using different data storage technologies to handle different data storage needs within a given software application.” James Serra, in his blog writes,  “Polyglot Persistence is a fancy term to mean that when storing data, it is best to use multiple data storage technologies, chosen based upon the way data is being used by individual applications or components of a single application.  Different kinds of data are best dealt with different data stores. “

Read More

When and Where to Use ETL Tools

July 17, 2018
Data is the foundation of all modern software. Without data, what’s the point? Now , what happens when we need to move that data from one application and/or database to another? That’s where ETL tools come into play. ETL stands from Extract, Transform, Load and essentially means data migration from one system to another. Sometimes these tools are in constant operation, sitting between various applications to manage continuous data migration. Many times these tools can be over-utilized and implemented in places they really don’t belong. My vision for ETL has always been a one - time thing, used exclusively to migrate data from one system to the other, not as an integration tool. However, there are plenty of use cases when ETL is the best option . T his blog addresses some existing options out there and some suggestions of how to improve your landscape.
Read More