I Don’t Want to Code SQL in a Notebook

At Aginity, we are lucky to interact with Data Analysts, Data Engineers and Data Scientists from across a wide range of industries and company sizes. It gives us a birds-eye view of some trends in data processing and the thousands of users of our SQL query applications...

At Aginity, we are lucky to interact with Data Analysts, Data Engineers and Data Scientists from across a wide range of industries and company sizes. It gives us a birds-eye view of some trends in data processing and the thousands of users of our SQL query applications (Aginity Pro and Aginity Team) are continually sharing feedback about features they need to succeed as well as sharing stories about the value they already find in some of the unique capabilities we offer, such as an active, shareable, executable SQL catalog.

One trend we hear a lot about is the proliferation of notebook solutions for analysts and data scientists. Whether that’s the open-source Zeppelin or Jupyter or any of the commercial products that wrap these in other functionality, they are all the rage as organizations attempt to provide their data workers with flexible tooling to deliver analytic solutions and products.

We like notebooks for a lot of the reasons they’ve been so successful. It’s great to have a framework that allows for multiple analytic languages (including SQL) to be combined into a “pipeline” and the embedded visualization and inline documentation helps to share not just the code but meaning, context and insights. We LOVE meaning, context, and sharing…more on that in a minute.

But where we hear a lot of our customers and users being challenged to adopt these, it’s for SQL-only or SQL-first use cases. We may be biased (I’ll admit it, we’re SQL peeps at the end of the day), but the fact is that notebooks don’t have many of the benefits a traditional SQL IDE or Query application offers to data analysts or to engineers/scientists who require SQL to initially find and prep data. Some common themes we hear are:

  • No good object browser to find data sets and limited visibility into data relationships between them
  • No re-use of SQL logic across notebooks, projects, or other applications and use cases
  • Few easy ways to transfer resulting data to other applications
  • No or limited autocomplete to make writing queries fast and accurate
  • No “accelerators” you find in typical IDEs (SQL generation, result set manipulation, etc.)

We think the unique approach we’ve taken in our next-gen SQL offerings provides the capabilities needed to solve these challenges in a familiar workflow to what most data professionals use today. And we’ve added unique features to allow for the capture of meaning, context and the sharing and re-use of SQL in a way that is far faster than notebooks for SQL-first and SQL-only use cases. And all of this is still to say that nothing prevents a user from taking their SQL assets from Aginity and using them in a notebook… but with the advantages of faster development and easy re-use for the next analysis or project.

In our approach, the familiar SQL IDE is enhanced with an embedded “Catalog” which enables the storage, searching, and re-use of metadata-enhanced SQL queries and snippets. This “write-once, use anywhere” approach to SQL logic accelerates development times and ensures that SQL authors can quickly find their “golden” logic for prepping data sets of the kind most useful to data scientists. And when your work is organized, cohesive, and instantly searchable— we offer a way to publicize this so your team members can find and reuse it, too. Our approach also ensures that the logic is easily transported to other applications, not just notebooks. BI developers, business analysts using Excel and others can all find and re-use SQL assets quickly and intuitively.

We’re thrilled to see analysts everywhere starting to knock-down the traditional barriers to communication, leaving room for unrestricted innovation. And for SQL-only or SQL-first use cases, we’re happy to be a critical component in our users’ success. For anyone who’s chafing at the SQL experience in notebooks, we’d recommend that it’s time to enter a modern new era of analytics with Aginity.

Share on linkedin
LinkedIn
Share on email
Email
Share on twitter
Twitter
Share on facebook
Facebook