{"id":926,"date":"2023-06-09T21:40:58","date_gmt":"2023-06-09T21:40:58","guid":{"rendered":"https:\/\/bandittracker.com\/?p=926"},"modified":"2023-06-30T15:10:18","modified_gmt":"2023-06-30T15:10:18","slug":"what-is-a-databricks-notebook","status":"publish","type":"post","link":"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/","title":{"rendered":"What Is A Databricks Notebook? (Explained)"},"content":{"rendered":"\n<p>If you\u2019re used to traditional code editors like Visual Studio or even VBA macros in Excel, you may find that notebooks take a little getting used to. I did!<\/p>\n\n\n\n<p>A Databricks notebook is a web-based interface to an interactive computing environment. Notebooks are used to create and run code, visualize results, narrate data stories, and collaborate with others.<\/p>\n\n\n\n<p>The goal is to allow data engineers and data scientists to work together in an interactive manner.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_64 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-66108726d37cc\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-66108726d37cc\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#How_Notebooks_Are_Different_To_Code_Editors\" title=\"How Notebooks Are Different To Code Editors\">How Notebooks Are Different To Code Editors<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#How_To_Use_A_Databricks_Notebook\" title=\"How To Use A Databricks Notebook\">How To Use A Databricks Notebook<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#How_Notebooks_Fit_Within_the_Databricks_Platform\" title=\"How Notebooks Fit Within the Databricks Platform\">How Notebooks Fit Within the Databricks Platform<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Collaborating_With_Databricks_Notebooks\" title=\"Collaborating With Databricks Notebooks\">Collaborating With Databricks Notebooks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Four_Types_Of_Visualizations_With_Notebooks\" title=\"Four Types Of Visualizations With Notebooks\">Four Types Of Visualizations With Notebooks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Automating_And_Scheduling_Features\" title=\"Automating And Scheduling Features\">Automating And Scheduling Features<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Version_Control_Is_Limited\" title=\"Version Control Is Limited\">Version Control Is Limited<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Origins_Of_Notebooks\" title=\"Origins Of Notebooks\">Origins Of Notebooks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/#Why_Use_Databricks_Instead_Of_Jupyter_Or_Google_Colab\" title=\"Why Use Databricks Instead Of Jupyter Or Google Colab?\">Why Use Databricks Instead Of Jupyter Or Google Colab?<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_Notebooks_Are_Different_To_Code_Editors\"><\/span>How Notebooks Are Different To Code Editors<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Here are some key differences for developers to be aware of.<\/p>\n\n\n\n<p>A key feature of Databricks is that you can write code in <a href=\"https:\/\/bandittracker.com\/what-languages-does-databricks-use\/\" data-type=\"post\" data-id=\"1009\">different languages within the same notebook<\/a>. You don&#8217;t do that with traditional code editors.&nbsp;<\/p>\n\n\n\n<p>Another departure is the ability to intersperse code and formatted text in markdown. This allows the analysts to add commentary and graphs to the code.<\/p>\n\n\n\n<p>You also get typical word processing features like auto-saving and revision history.<\/p>\n\n\n\n<p>Built-in visualization is another key feature. When you run code that produces plots or charts, these are displayed directly beneath the code section.<\/p>\n\n\n\n<p>This picture shows a SQL query and the output beneath it in a bar chart format. Another user could choose to change the visualization to a pie chart.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"500\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-sql-statement-and-chart-1.jpg\" alt=\"\" class=\"wp-image-928\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-sql-statement-and-chart-1.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-sql-statement-and-chart-1-300x250.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<p>Developers can also create dashboards from notebooks. This provides a handy way to share insights and findings with non-technical stakeholders.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_To_Use_A_Databricks_Notebook\"><\/span>How To Use A Databricks Notebook<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Databricks notebooks are made up of several components that allow you to write code, execute commands, visualize data, and document your work. Here&#8217;s a breakdown of the key parts:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Cells<\/h3>\n\n\n\n<p>A notebook is divided into segments called &#8220;cells&#8221;. These are the basic units where you can input code or text. There are two main types of cells: code cells and markdown cells.<\/p>\n\n\n\n<p>Code cells are where you write and execute your code. When you run or execute the cell, the results are displayed directly below the cell.<\/p>\n\n\n\n<p>They can be run individually, in groups, or all at once.<\/p>\n\n\n\n<p>Markdown cells allow you to input text using markdown. This is a lightweight markup language, in other words it\u2019s like HTML (but different).<\/p>\n\n\n\n<p>You use markdown cells to add narrative such as headings and subheadings. You can also create lists, ad links to web pages, and even images or videos.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"500\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-markdown-and-code-cell.jpg\" alt=\"\" class=\"wp-image-937\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-markdown-and-code-cell.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-markdown-and-code-cell-300x250.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<h3 class=\"wp-block-heading\">Output Area<\/h3>\n\n\n\n<p>The output area appears below each code cell, displaying the results of the code execution.<\/p>\n\n\n\n<p>This can include printed text, tables, data frames, plots, or even interactive visualizations, depending on the code you&#8217;ve written.<\/p>\n\n\n\n<p>The picture below shows the output of a SQL statement that returns ten rows from a table.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"500\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-code-cell-and-output.jpg\" alt=\"\" class=\"wp-image-936\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-code-cell-and-output.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-code-cell-and-output-300x250.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<p>The output area is also where you&#8217;ll see error messages. When you&#8217;re trying to fix issues with your code, check out our article on <a href=\"https:\/\/bandittracker.com\/debug-databricks-notebooks\/\" data-type=\"post\" data-id=\"1021\">debugging Databricks notebooks<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Basic Editing<\/h3>\n\n\n\n<p>If you like to jump straight into things without reading the manual, I\u2019ll give you some pointers to the not-so-obvious editing features.<\/p>\n\n\n\n<p>You can use the edit drop-down menu to add cells. If you intend to be a regular notebook user, I advise that you learn and practice the short-cuts for these simple actions.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"500\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-edit-add-cells.jpg\" alt=\"\" class=\"wp-image-935\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-edit-add-cells.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/databricks-notebook-edit-add-cells-300x250.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<p>To edit an existing cell, double-click inside it.<\/p>\n\n\n\n<p>Use the dropdown menu at the top right of each cell to tell the Notebook whether you are writing markdown, SQL, Python, or another language. If you try to run SQL in a Python cell, you\u2019ll get a syntax error.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_Notebooks_Fit_Within_the_Databricks_Platform\"><\/span>How Notebooks Fit Within the Databricks Platform<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In the Databricks environment, notebooks connect to a computational cluster that can be scaled up or down according to the requirements of the task at hand.<\/p>\n\n\n\n<p>The platform integrates with various data sources (e.g. S3 or Snowflake) and supports multiple data formats.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"400\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/snowflake-deltalake-s3-azure.jpg\" alt=\"\" class=\"wp-image-921\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/snowflake-deltalake-s3-azure.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/snowflake-deltalake-s3-azure-300x200.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<p>Notebooks natively support several languages, including Python, Scala, SQL, and R. You can even switch between languages in a single notebook.<\/p>\n\n\n\n<p>Notebooks are also integrated with many popular data science libraries and frameworks, such as TensorFlow, PyTorch, Scikit-learn.<\/p>\n\n\n\n<p>These features allow users to perform exploratory data analysis, build machine learning models, and create rich visualizations directly within their notebooks.<\/p>\n\n\n\n<p>Multiple users can sign into the Databricks platform and work on a notebook simultaneously. They see each other&#8217;s inputs in real time, as they would with something like Google Docs.<\/p>\n\n\n\n<p>In terms of automation, Databricks notebooks can be scheduled to run as jobs. This is a handy feature for reports that need to be generated on a regular schedule.<\/p>\n\n\n\n<p>They can also be exported in various formats, such as HTML, PDF, or even Jupyter Notebooks.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Collaborating_With_Databricks_Notebooks\"><\/span>Collaborating With Databricks Notebooks<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Notebooks allow real time collaboration within the same notebook. It\u2019s like working with a souped-up version of Google Docs.<\/p>\n\n\n\n<p>If you\u2019re wondering how that doesn\u2019t create chaos, here are my top three tips for effective collaboration.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Make Use Of Commenting<\/h3>\n\n\n\n<p>Databricks notebooks support commenting within cells. <\/p>\n\n\n\n<p>You can leave comments on a particular cell, and other users can respond to these comments. This helps reducing confusion.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Revision History Reduces Conflict<\/h3>\n\n\n\n<p>Databricks notebooks automatically save changes and keep a history of revisions. You can revert to a previous version of the notebook if necessary.<\/p>\n\n\n\n<p>This is particularly useful when collaborating with others, as it allows you to track changes made by different contributors and rollback changes if needed. It can stop a few office arguments!<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"400\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/developer-analyst-conflict.jpg\" alt=\"\" class=\"wp-image-932\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/developer-analyst-conflict.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/developer-analyst-conflict-300x200.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<h3 class=\"wp-block-heading\">3. Apply Permissions<\/h3>\n\n\n\n<p>You can set permissions at the notebook level. These control who can view, edit, or manage the notebook. This can be configured for individual users or user groups.<\/p>\n\n\n\n<p>So, if you have a manager that you don\u2019t want to \u201chelp\u201d with your notebook (i.e. break it), restrict them to viewing privileges.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Four_Types_Of_Visualizations_With_Notebooks\"><\/span>Four Types Of Visualizations With Notebooks<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>One of the strengths of Databricks notebooks lies in their powerful visualization capabilities. This is what makes them popular with data scientists.<\/p>\n\n\n\n<p>Here are four types of visualizations you can achieve:<\/p>\n\n\n\n<ul>\n<li>Tables and basic charts<\/li>\n\n\n\n<li>Complex charts from external plotting libraries<\/li>\n\n\n\n<li>Interactive visualizations<\/li>\n\n\n\n<li>Dashboards<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Tables and In-Built Charts<\/h3>\n\n\n\n<p>When a command in a Databricks notebook returns a data table, the notebook automatically provides a user-friendly interface for visualizing this data.<\/p>\n\n\n\n<p>A tab appears beneath the code cell displaying the table, with options to choose between various plot types like bar charts, line charts, scatter plots, area charts, etc.<\/p>\n\n\n\n<p>You don\u2019t get that with most code editors.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Integration with Plotting Libraries<\/h3>\n\n\n\n<p>Databricks notebooks support a wide range of external visualization libraries. That includes Matplotlib, Seaborn, ggplot2, Bokeh, and Plotly.<\/p>\n\n\n\n<p>This provides flexibility for more complex or customized visualizations.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Interactive Visualizations<\/h3>\n\n\n\n<p>Databricks notebooks are compatible with libraries that generate interactive plots.<\/p>\n\n\n\n<p>For example, Plotly and Bokeh allow the creation of plots with zoom, pan, hover-to-highlight, and click-to-select functionalities.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Dashboards<\/h3>\n\n\n\n<p>A dashboard is a collection of visualizations, tables, and other views within a single notebook.<\/p>\n\n\n\n<p>These dashboards can be shared with other users and stakeholders, allowing a suite of analytics to be delivered in a single package.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Automating_And_Scheduling_Features\"><\/span>Automating And Scheduling Features<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"400\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/schedule.jpg\" alt=\"\" class=\"wp-image-933\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/schedule.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/schedule-300x200.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<p>As part of the Databricks platform, you also get access to automation and scheduling features. Remember that your notebook runs on a computing environment. <em>If you want more details of what&#8217;s under the hood, check out our article on <a href=\"https:\/\/bandittracker.com\/what-are-databricks-clusters\/\" data-type=\"post\" data-id=\"918\">databricks clusters.<\/a><\/em><\/p>\n\n\n\n<p>Aside from clusters, the platform terminology includes:<\/p>\n\n\n\n<ul>\n<li>Jobs<\/li>\n\n\n\n<li>Workflows<\/li>\n\n\n\n<li>Alerts<\/li>\n\n\n\n<li>Automation APIs<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Jobs<\/h3>\n\n\n\n<p>In Databricks, you can create a \u201cjob\u201d from a notebook that you\u2019ve packaged with parameters to execute.<\/p>\n\n\n\n<p>You can schedule jobs to run on a schedule or on a trigger (e.g., in response to an API call). This can be useful for daily reports, or for triggering a script after an ETL process finishes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Workflows<\/h3>\n\n\n\n<p>Databricks notebooks support calling other notebooks as part of a workflow. This lets you break down complex analyses into manageable, reusable parts.<\/p>\n\n\n\n<p>You can have one &#8216;master&#8217; notebook that orchestrates other notebooks, each performing a specific part of the overall analysis.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Alerts<\/h3>\n\n\n\n<p>You can configure Databricks to send you an alert if a scheduled job fails, completes, or takes longer than expected to run. The alert can be an email, a text message, or even a call.<\/p>\n\n\n\n<p>This can help you ensure that your automated tasks are running smoothly.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">APIs<\/h3>\n\n\n\n<p>Databricks provides a REST API that you can use to create, manage, and run jobs programmatically.<\/p>\n\n\n\n<p>This can be useful if you want to trigger Databricks jobs from external systems. They also come into play when you&#8217;re managing a lot of jobs and want to automate the process.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Version_Control_Is_Limited\"><\/span>Version Control Is Limited<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Version control allows teams to track and manage changes to their code. Databricks notebooks provide several features to support version control. However, this is quite limited when compared to traditional programming.<\/p>\n\n\n\n<p>Basic versioning is provided by a notebook\u2019s revision history. You can view the revision history, revert to a previous version, or make a new copy of the notebook at a specific point in the history.<\/p>\n\n\n\n<p>Most developers will be familiar with using git or a similar system for version control that allows sophisticated control through commits, pushing and pulling changes, and creating and merging branches.<\/p>\n\n\n\n<p>Databricks introduced their Repos system to offer similar features. However, Repos don\u2019t (yet) offer advanced git functionality like resolving conflicts.<\/p>\n\n\n\n<p>If you&#8217;re an analyst or a manager, don&#8217;t be surprised if you get some push-back from developers about this aspect of Databricks. Bad source control can drive them crazy (here&#8217;s an <a rel=\"noreferrer noopener\" href=\"https:\/\/medium.com\/@seade03\/please-dont-make-me-use-databricks-notebooks-3d07a4a332ae\" target=\"_blank\">amusing but educational <\/a><a rel=\"noreferrer noopener\" href=\"https:\/\/medium.com\/@seade03\/please-dont-make-me-use-databricks-notebooks-3d07a4a332ae\" target=\"_blank\">t<\/a><a rel=\"noreferrer noopener\" href=\"https:\/\/medium.com\/@seade03\/please-dont-make-me-use-databricks-notebooks-3d07a4a332ae\" target=\"_blank\">a<\/a><a rel=\"noreferrer noopener\" href=\"https:\/\/medium.com\/@seade03\/please-dont-make-me-use-databricks-notebooks-3d07a4a332ae\" target=\"_blank\">k<\/a><a href=\"https:\/\/medium.com\/@seade03\/please-dont-make-me-use-databricks-notebooks-3d07a4a332ae\" target=\"_blank\" rel=\"noreferrer noopener\">e<\/a> on the topic).<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"400\" src=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/angry-developer.jpg\" alt=\"\" class=\"wp-image-934\" srcset=\"https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/angry-developer.jpg 600w, https:\/\/bandittracker.com\/wp-content\/uploads\/2023\/06\/angry-developer-300x200.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Origins_Of_Notebooks\"><\/span>Origins Of Notebooks<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Databricks certainly didn\u2019t invent the concept of a notebook. If you don\u2019t want a quick history lesson, skip this section.<\/p>\n\n\n\n<p>The first appearance arose from Mathematica, an environment and programming language used in STEM fields for complex maths analysis. <\/p>\n\n\n\n<p>Its developers rolled out \u201ccomputational documents\u201d in 2007 as a code editor with added commentary and reports. This was an early implementation of what we call \u201cnotebooks\u201d now.<\/p>\n\n\n\n<p>But Mathematica was and is rather niche in terms of programming environments. The general introduction to a wider base of programmers was with the development of Jupyter.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">iPython -&gt; Jupyter<\/h3>\n\n\n\n<p>Jupyter\u2019s early name was iPython. It was an open-source project started in 2001 to create a better interactive experience for Python developers. It rolled out a notebook interface in 2005 that drew a lot from Mathematica but was still quite primitive.<\/p>\n\n\n\n<p>2011 was a key year. The notebook interface expanded to allow code, text, mathematical expressions, and visualizations in the same document.<\/p>\n\n\n\n<p>The iPython team spun out a separate project called Jupyter in 2014 to provide a language-agnostic environment. That was the start of the classic <a href=\"https:\/\/jupyter.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Jupyter<\/a> Notebook that offers a bundle of tools in one web-based document:<\/p>\n\n\n\n<ul>\n<li>Code editor<\/li>\n\n\n\n<li>Rich text editor<\/li>\n\n\n\n<li>Terminal<\/li>\n\n\n\n<li>File browser<\/li>\n\n\n\n<li>Rich outputs<\/li>\n<\/ul>\n\n\n\n<p>Notebooks in Databricks are based on the Jupyter Notebook.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_Use_Databricks_Instead_Of_Jupyter_Or_Google_Colab\"><\/span>Why Use Databricks Instead Of Jupyter Or Google Colab?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>I\u2019ve already talked a bit about Jupyter which is still popular. You should also be aware of <a href=\"https:\/\/colab.research.google.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Google Colab<\/a> which is the mighty Google\u2019s version of a notebook platform.<\/p>\n\n\n\n<p>In this section, I\u2019ll discuss some pros and cons if you\u2019re choosing between the three platforms.<\/p>\n\n\n\n<p>To get started with Jupyter notebooks, you usually need to install the Python notebook package on your local machine or server. This isn\u2019t difficult for Python developers. But it may be a little daunting for data scientists without a programming background.<\/p>\n\n\n\n<p>The advantage of both Databricks and Google Colab is that the environment is provided and serviced for you. You just need an account on the platform.<\/p>\n\n\n\n<p>So, let\u2019s talk about accounts. For Colab, all you need is a free Google account (e.g. a Gmail account) to run code and visualizations.<\/p>\n\n\n\n<p>Google largely provide their platform as a free service. Jupyter is also free. In contrast, Databricks provides the community edition for free, but it\u2019s very limited in usage.<\/p>\n\n\n\n<p>But there\u2019s no such thing as a free mega-lunch\u2026I mean mega-loads. Databricks was designed from the start to handle large data volumes. That\u2019s what you\u2019re paying for. You\u2019ll struggle to run big data projects with Jupyter or Colab.<\/p>\n\n\n\n<p>The Databricks platform also offers powerful job scheduling and automated features that are built in to the system. With Jupyter, you\u2019ll need to use external tools for this. Colab also doesn\u2019t have these capabilities.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you\u2019re used to traditional code editors like Visual Studio or even VBA macros in Excel, you may find that notebooks take a little getting used to. I did! A Databricks notebook is a web-based interface to an interactive computing environment. Notebooks are used to create and run code, visualize results, narrate data stories, and &#8230; <a title=\"What Is A Databricks Notebook? (Explained)\" class=\"read-more\" href=\"https:\/\/bandittracker.com\/what-is-a-databricks-notebook\/\" aria-label=\"More on What Is A Databricks Notebook? (Explained)\">Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[29],"tags":[],"_links":{"self":[{"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/posts\/926"}],"collection":[{"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/comments?post=926"}],"version-history":[{"count":5,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/posts\/926\/revisions"}],"predecessor-version":[{"id":1028,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/posts\/926\/revisions\/1028"}],"wp:attachment":[{"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/media?parent=926"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/categories?post=926"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bandittracker.com\/wp-json\/wp\/v2\/tags?post=926"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}