chore: import upstream snapshot with attribution
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
name: my-env
|
||||
channels:
|
||||
- conda-forge
|
||||
dependencies:
|
||||
- python=3.9
|
||||
- pip
|
||||
- pip:
|
||||
- mlflow
|
||||
- azureml-core
|
||||
- azure-ai-ml>=1.12.0
|
||||
- azureml-dataset-runtime[pandas,fuse]
|
||||
- azureml-telemetry
|
||||
- mltable>=1.2.0
|
||||
- pandas
|
||||
- pillow
|
||||
@@ -0,0 +1,21 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
# Licensed under the MIT license.
|
||||
|
||||
import os
|
||||
import pandas as pd
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--input_data_file", type=str)
|
||||
parser.add_argument("--output_data_folder", type=str)
|
||||
|
||||
args, _ = parser.parse_known_args()
|
||||
|
||||
input_df = pd.read_json(args.input_data_file, lines=True)
|
||||
|
||||
# data preparation, e.g. data sampling, data cleaning, etc.
|
||||
processed_data = input_df.sample(n=20, replace=True, random_state=1)
|
||||
|
||||
# export data into output folder
|
||||
output_file_path = os.path.join(args.output_data_folder, "processed_data.csv")
|
||||
processed_data.to_csv(output_file_path, index=False, header=True)
|
||||
@@ -0,0 +1,26 @@
|
||||
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
|
||||
|
||||
type: command
|
||||
|
||||
name: data_prep
|
||||
display_name: data preparation
|
||||
version: 0.0.1
|
||||
|
||||
inputs:
|
||||
input_data_file:
|
||||
type: uri_file
|
||||
|
||||
outputs:
|
||||
output_data_folder:
|
||||
type: uri_folder
|
||||
|
||||
code: ./
|
||||
|
||||
environment:
|
||||
image: mcr.microsoft.com/azureml/inference-base-2004:latest
|
||||
conda_file: ./conda.yaml
|
||||
|
||||
command: >-
|
||||
python data-prep.py
|
||||
--input_data_file ${{inputs.input_data_file}}
|
||||
--output_data_folder ${{outputs.output_data_folder}}
|
||||
Reference in New Issue
Block a user