HRD University

Creating Azure Databricks Workload

Introduction to Azure Databricks and Creating First Cluster

Creating Cluster in Databricks community Edition

Creating First Notebook

DBFS

DBFS UI

Creating Documentation Cell

Uploading file in DBFS

Reading File from DBFS and Creating First DataFrame

End of Module

Spark Documentation

Creating PySpark DataFrame using list of Rows

Creating PySpark DataFrame using explicit schema

Creating PySpark DataFrame using pandas DataFrame

Creating DataFrame Ex 1

Creating DataFrame Ex2

Creating DataFrame using PySpark Datatypes

Creating DataFrame Ex3

Creating DataFrame Ex4

Documentation on Dataframe functions (Spark)

Select-1 & 2

Select dataframe functions with alias

withColumnRenamed

withColumn

DataFrame Function-Count

DataFrame Function-Limit

Describe DataFrame

Dtypes and printschema

tail() and take() function in databricks

Show or Display

Functions

Col, Lit and Concat Functions

Renaming Column Names

Select & withcolumn

Filter

Filtering Nulls

Select & concat

Sort

Datagram Function-Sort in Databricks

DataFrame Function-Sort with Multiple Column in Databricks

DataFrame Function-Drop in Databricks

Drop Duplicates

Handling Null using dropna or na.drop in Pyspark _Databricks Tutorial for Beginners_Azure Databricks

DataFrame Function - Group by _ Aggregation in Databricks

Run a Databricks notebook from another notebook

Introduction to Delta Lake

Internals of Delta Lake

Optimize in Delta Lake

Time Travel in delta lake

UpSert in Delta Lake

Z ordering in Delta Lake

E2E Capstone

Azure Databricks intro

Azure Intro and ADLS

Big Data File Formats

CSV to delta table

Jobs in Databricks

Method Using Secret Key and Service Principal

Unmount

Reading Json(Constructor)and Writing into Parquet

Reading Json(driver)and Writing into Parquet

Reading CSV with user Schema

Complex Json

Reading Excel

Reading CSV and transforming

Creating User Defined Schema

Date and Time Stamp Functions

Handling Null Values

Views

Why Create User Defined Schema