thebernardlim

Articles on Cloud and Code

Azure Data Engineer Associate Certification Study Notes

Azure Data Engineer Associate certification study notes

Azure Data Engineer Associate Certification Study Notes Azure Databricks Cluster Modes Standard Terminate after 120 seconds Python, R, Scala, SQL For single users ...

Posted by thebernardlim on January 28, 2020

Azure IOT Hubs vs Event Hubs - What are the differences?

Quick summary on the differences between IoT Hubs vs Event Hubs

Both Event Hub and IOT Hub are used for stream ingestion. So when do we use which? Below is a quick summary: Difference #1 : Communication Event Hub: 1 way communication from devices / sources ...

Posted by thebernardlim on January 12, 2020

Azure Data Factory - How to inspect schema of Avro files?

ADF is unable to infer schema of Avro files. Here is the solution.

Problem Avro file format is the only format when capturing events from Event Hub. However Avro files cannot be processed directly (To inspect schema) on Data Factory. Solution Use a Databricks ...

Posted by Bernard Lim on January 11, 2020