Microsoft Academic Graph (MAG) Azure Data Lake Store (ADLS) Distribution Preview
Please be advised that this distribution is in free preview stage. Pricing structure is subject to change.
Open Data License: ODC-BY
Attribution Guidelines:
When using Microsoft Academic Graph (MAG) in a product or service, or including MAG in a redistribution, please acknowledge MAG using the URI https://aka.ms/msracad. For publications and reports, please cite the following article:
Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June (Paul) Hsu, and Kuansan Wang. 2015. An Overview of Microsoft Academic Service (MA) and Applications. In Proceedings of the 24th International Conference on World Wide Web (WWW '15 Companion). ACM, New York, NY, USA, 243-246. DOI=http://dx.doi.org/10.1145/2740908.2742839
Creating Azure Data Lake Store and grant MS people to upload MAG dataset
1. Create an Azure Subscription
Please create a new Azure subscription for MAG. If your organization already using Azure, this could be a separate subscription under the same tenant id. If you start from scratch, for example “create Azure free account”, the subscription will be created under a new tenant id.
2. Create an Azure Data Lake Store
2.1. Home > Create a resource > Storage > Data Lake Storage Gen1 > enter name and create ADLS, say "mymagdls" (You need to choose a different name because it is unique among all Azure Data Lake Stores")
3. Create Azure Active Directory Group
3.1. Home > Azure Active Directory > Groups > All groups > "+ New group" > Say the group name is "MAG Writer"
Membership type: Assigned
Enable Office features? Yes
4. Add group as a contributor to ADLS
4.1. Home > "mymagdls" > Access control (IAM) > "+ Add"
Role: Controbutor
Assign access to: “Azure AD user, group, or application”
Select: “MAG Writer”
"Save"
5. Assign access permission for “MAG Writer” group to root folder of your ADLS.
5.1. Home > "mymagdls" > Data Explorer > Access > "+ Add"
· Select User or Group: "MAG Writer"
· Select Permissions
5.2. After saved, “MAG Writer” should be in the access list
- Click “Advanced”, you should see the “MAG Writer” are in the “Default Permissions”.
6. Invite Microsoft people to Active Directory as a guest user
6.1. Home > Azure Active Directory > Users > All Users > "+ New guest user"
Enter email (someone@microsoft.com) to the invite box.
7. Add Microsoft people to Active Directory Group
7.1. Azure Active Directory > All groups > "MAG Writer" > Members > Add members
> Select someone@microsoft.com > Select
o After saved, click on “Overview”, you should see 1 new user in the group.
o Click on “Members”, you should see someone@microsoft.com in the group.
8. Repeat step 6 and 7 if need to add more people
9. Create Service-to-service authentication with Data Lake Store using Azure Active Directory
Follow instruction to complete step 1 and 2.
9.1. Step 1: Create an Active Directory web application
- Create an application called “MAG Writer App”.
- Enter arbitrary URL for the “Sign-on URL” field
9.2. Step 2: Get application ID, authentication key, and tenant ID
- Take note of
o Application ID
o Authentication key
o Tenant ID
10. Assign permission for “MAG Writer App”
Assign access permission for “MAG Writer App” to both ADLS itself and the root folder of the ADLS (reference Assign the Azure AD application to the Azure Data Lake Store account file or folder)
10.1. Assign access permission for “MAG Writer App” to your ADLS.
Home > "mymagdls" > Access control (IAM) > "+ Add"
10.2. Assign access permission for “MAG Writer App” to root folder of your ADLS.
Home > "mymagdls" > Data Explorer > Access > "+ Add"
o Make sure “MAG Writer App” has Read/Write/Execute permission for both the ADLS itself and the root folder of the ADLS.
11. Send following information to the Microsoft contact
o ADLS name
o Application ID
o Authentication key
o Tenant ID