Who Uses Databricks? (Major Customers Explored)

In today’s fast-evolving technological landscape, data has been coined as the new oil. Databricks is a powerful data analytics platform at the heart of this data revolution. You may be wondering if this new platform has already been adopted by large well-known companies. The answer is an indisputable yes! This article looks at its far-reaching … Read more

How To Run A Job From A Databricks Notebook

A common Databricks task is to schedule a job that calls a notebook. However, sometimes you will want to do this in reverse. In other words, you want to run a notebook that launches a job. This is most likely when you have set up a job that you want to test without creating a … Read more

How To Debug Databricks Notebooks

Debugging is an essential part of the coding process. Even the most experienced programmers make mistakes. It’s through debugging that these mistakes are found and fixed. Each cell in a Databricks notebook has its own output, and any error messages are also displayed in the cell output. This is usually the starting point for your … Read more

What Languages Does Databricks Use? (Explained)

In Databricks, notebooks provide the ability to develop real-time machine learning, data enginering, and data analytics workflows. Notebooks support four programming languages: You can use one or all languages within a single notebook. If you’re starting out, you may be wondering if one of these languages is better for your purposes. Does the company favor … Read more

Delta Tables In Databricks (For Beginners)

Delta tables are a core feature of Databricks. This article assumes you are completely new to the platform but have some familiarity with SQL or Spark basics. Read on to get an understanding of what they are, how they differ from other storage formats, and how to create and use them. What Is A Delta … Read more

What SQL Does Databricks Use? (Explained)

The default SQL standard used by Databricks has changed since the platform first launched. Databricks originally used Spark SQL as the default SQL dialect, but changed the standard in late 2021. The default dialect in Databricks is currently ANSI Standard SQL. Because there are differences between the two dialects, developers and analysts should be aware … Read more

Reading And Writing To S3 With Databricks (Examples)

Before you start exchanging data between Databricks and S3, you need to have the necessary permissions in place. We have a separate article that takes you through configuring S3 permissions for Databricks access. The rest of this article provides code examples for common use cases when reading and writing data with Databricks and S3. Create … Read more

Configure Permissions And Roles For Databricks And S3

Permissions in AWS are defined by IAM roles (Identity and Access Management). To work with files in S3, your Databricks deployment will need an IAM role that has read/write permissions to an S3 bucket. You have two configuration options: The Unity Catalog is a new feature from Databricks this aims to simplify what you otherwise … Read more

Working With Databricks DBFS (For Beginners)

The local file system in Databricks is known as the DBFS. This article explains the underlying concepts of DBFS for Databricks beginners and people who are new to cloud storage. Once you’ve grasped the concepts, the article shows you how to: What Is Databricks DBFS? Even if you’re new to the Cloud, you will be … Read more

How To Call One Databricks Notebook From Another

You will often want to reuse code from one Databricks notebook in another. This step-by-step beginner guide shows you how to: If you’re new to this technology, don’t worry. I assume that you know the basics of notebooks in Databricks. But that’s all you need to follow along. Typical Use Case Here is a typical … Read more