Use the following steps to create a Hive query language (HiveQL) script that defines a query. If your workflow used other resources, such as a jar that contains a MapReduce application, you need to add those resources as well. Verify the actual JDBC driver that exists at /usr/share/java/. For more information about Sqoop, see Use Apache Sqoop with HDInsight. Ī Sqoop action exports the contents of the new Hive table to a table created in Azure SQL Database. The Hive script used in this document counts the total visits for each platform, such as Android or iPhone, and stores the counts to a new Hive table.įor more information about Hive, see.

The record format appears like the following text: 8 18:54:20 en-US Android Samsung SCH-i500 California United States 13.9204007 0 0Ģ3 19:19:44 en-US Android HTC Incredible Pennsylvania United States NULL 0 0Ģ3 19:19:46 en-US Android HTC Incredible Pennsylvania United States 1.4757422 0 1 Each row of data describes a visit from a specific mobile device. Actions are definitions for tasks, such as running Hive, Sqoop, MapReduce, or other processes:Ī Hive action runs an HiveQL script to extract records from the hivesampletable that's included with HDInsight. The workflow used in this document contains two actions. If secure transfer is enabled for Azure Storage, the URI would be wasbs://. wasb:// for Azure Storage, abfs:// for Azure Data Lake Storage Gen2 or adl:// for Azure Data Lake Storage Gen1. The URI scheme for your clusters primary storage. This article uses a database named oozietest. See Create a database in Azure SQL Database in the Azure portal.

See Connect to HDInsight (Apache Hadoop) using SSH.Īn Azure SQL Database. See Get Started with HDInsight on Linux.Īn SSH client. PrerequisitesĪ Hadoop cluster on HDInsight. To use Oozie on clusters with Enterprise Security Package please see Run Apache Oozie in HDInsight Hadoop clusters with Enterprise Security Package. To learn more about Data Factory, see Use Apache Pig and Apache Hive with Data Factory. Another option to define workflows with HDInsight is to use Azure Data Factory.

Cloudera docker on mac