20. Create AI Model using ML Flow in Microsoft Fabric | скачать в хорошем качестве

20. Create AI Model using ML Flow in Microsoft Fabric |

In this video, I discussed about creating AI model using ML flow in Microsoft Fabric. Below are the steps to create model in Fabric 1. Create a workspace 2. Create a notebook 3. Get the data 4. Prepare the data 5. Train Machine learning models 6. Explore your experiments 7. Save the Model 8. Save the notebook We will work with the diabetes dataset from azure open datasets. Diabetes dataset - Azure Open Datasets | Microsoft Learn Step 1: Get data - Diabetes(Azure open Dataset) get it and load in dataframe. Azure storage access info for open dataset diabetes blob_account_name = "azureopendatastorage" blob_container_name = "mlsamples" blob_relative_path = "diabetes" blob_sas_token = r"" # Blank since container is Anonymous access Set Spark config to access blob storage wasbs_path = f"wasbs://%s@%s.blob.core.windows.net/%s" % (blob_container_name, blob_account_name, blob_relative_path) spark.conf.set("fs.azure.sas.%s.%s.blob.core.windows.net" % (blob_container_name, blob_account_name), blob_sas_token) print("Remote blob path: " + wasbs_path) # Spark read parquet, note that it won't load any data yet by now df = spark.read.parquet(wasbs_path) display(df) Step 2: Build Box Plot on Y axis. To identity Q1(25%) and Q3(75%) percentiles. Step 3: Convert datafram to Pandas DataFrame df = df.toPandas() df.head() Step 4: Describe dataframe to see 75% value for Y. df.describe() Step 5: Add 'Risk' Column, to classify low risk or high risk. Accordingly we can apply classification algorithm for training. df['Risk'] = (df['Y'] 212).astype(int) Step 6: Split data for training and testing. Using scikit-learn library from sklearn.model_selection import train_test_split X, y = df_clean[['AGE','SEX','BMI','BP','S1','S2','S3','S4','S5','S6']].values, df_clean['Risk'].values X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0) Step 7: create experiment using ML Flow. import mlflow experiment_name = "diabetes-classification" mlflow.set_experiment(experiment_name) Step 8: Train the model using scikit-learn library from sklearn.linear_model import LogisticRegression with mlflow.start_run(): mlflow.sklearn.autolog() model = LogisticRegression(C=1/0.1, solver="liblinear").fit(X_train, y_train) Step 9: Explore your experiment from Workspace. Observe Run metrics values. Step 10: Save the Model. Step 11: Save the notebook. #microsoftfabric #mlflow #ai #datascience #fabric #microsoft #azure #dataengineering #learning