thebernardlim

Articles on Cloud, Code & M365

Azure Data Engineer Associate (DP-200, DP-201) Certification Study Guide

Sharing my experience to getting certified for the Azure Data Engineer Associate certification

I had just recently cleared both DP-200 and DP-201 exams for the Azure Data Engineer Associate certification. Below are the list of steps I undertook to achieve them. Hope you will find them useful...

Posted by thebernardlim on February 3, 2020

Azure Streaming Analytics - Export / Import Streaming Analytics Jobs

Export / Import Streaming Analytics using Visual Studio Code

Unlike other services, there is no ‘Export Template’ option within the portal as shown in an example below: Exporting One way to export a Streaming Analytics job is through Visual Studio...

Posted by thebernardlim on January 28, 2020

Azure Data Engineer Associate Certification Study Notes

Azure Data Engineer Associate certification study notes

Azure Data Engineer Associate Certification Study Notes Azure Databricks Cluster Modes Standard Terminate after 120 seconds Python, R, Scala, SQL For single users ...

Posted by thebernardlim on January 28, 2020

Azure IOT Hubs vs Event Hubs - What are the differences?

Quick summary on the differences between IoT Hubs vs Event Hubs

Both Event Hub and IOT Hub are used for stream ingestion. So when do we use which? Below is a quick summary: Difference #1 : Communication Event Hub: 1 way communication from devices / sources ...

Posted by thebernardlim on January 12, 2020

Azure Data Factory - How to inspect schema of Avro files?

ADF is unable to infer schema of Avro files. Here is the solution.

Problem Avro file format is the only format when capturing events from Event Hub. However Avro files cannot be processed directly (To inspect schema) on Data Factory. Solution Use a Databricks ...

Posted by Bernard Lim on January 11, 2020