Technology: Microsoft Azure
Intro into Microsoft Azure
In the current digital age, many companies offer their services online, leading to an increase in the amount of data available. This data can be useful for gaining insights into various business processes. To analyse this data, the use of data analytics solutions has grown significantly in recent years. Finaps utilises several of these solutions, including Azure, created by Microsoft as their public cloud platform. In this article, we will discuss all the data-related possibilities of this platform.
What is Microsoft Azure?
Azure offers a wide range of services, including hosting, networking, security, and data-related products. These products include Platform as a Service (PaaS), Infrastructure as a Service (IaaS), and managed database service capabilities (https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/get-started/what-is-azure). The cloud service operates on a pay-as-you-go model, in which users only pay for the hours and services they actively use. Some services offer an easy-to-use user interface for business developers, while others provide a collaboration platform that supports open-source languages. Azure’s services are diverse, and some are discussed in more detail below. For a comprehensive description of these products, refer to the Microsoft documentation.
What can Azure do?
In the entire collection of services, lots of data-related products are available. Together they can make up all the steps of the data analytics lifecycle. From data warehousing and ETL (Extract, Transform and Load) to advanced machine learning.
Data Storage
An important first step for a company is to find a data storage solution. Azure offers several products that allow users to store both structured and unstructured data, such as Azure SQL database, Azure Cosmos DB, and Health Data Services. For a more durable solution, Azure Data Lake Storage can be utilized.
Data Engineering
Using other products, a proper ETL flow can be created to ingest data from external sources in the above storage options. An example of such a product is Azure Data Factory. This service allows regular data extractions from several external databases. Then, basic data transformations can be performed, and the resulting data can be loaded into one of your preferred data storage solutions. Another product that can be used is Azure Synapse Analytics. However, here other functionalities, besides ETL, can be used as well.
Data Analytics
Once your data is stored securely, it can be analysed to discover insights and patterns. Azure offers several products that allow developers to use low-code or multiple programming languages for this purpose, such as Azure Databricks, Azure HDInsights, Azure Synapse Analytics, Azure Analysis Services, and Azure Data Explorer. Databricks provides a collaborative environment for Apache Spark development, while HDInsights enables the use of cloud clusters such as Hadoop, Spark, R Server, HBase, and Storm. These tools offer various options for data analysis.
Data Science
Incorporating Machine Learning and other advanced services into your data analytics lifecycle can be beneficial. Azure offers several products that allow users to do this, including Azure Machine Learning and more specialized products such as Azure Bot Service, Microsoft Genomics, Azure Video Indexer, and Azure Form Recognizer. These tools focus on advanced topics like text and video analysis, and chatbots. Azure Cognitive Services is a product that enables users without extensive AI or ML knowledge to utilize these techniques.