Add Python API for all CLI commands with WorkflowOutput returns

[Copilot]

This PR implements a comprehensive Python API for all Olive CLI commands, allowing users to programmatically execute workflows and receive structured WorkflowOutput objects containing ModelOutput instances.

Overview

The new Python API provides 13 functions corresponding to all CLI commands:

Workflow Functions (return WorkflowOutput with ModelOutput instances):

• auto_opt() - Auto-optimize models for performance
• finetune() - Fine-tune models using LoRA/QLoRA
• quantize() - Quantize models for reduced size
• capture_onnx() - Capture ONNX graphs from PyTorch models
• generate_adapter() - Generate adapters for ONNX models
• session_params_tuning() - Tune ONNX Runtime parameters
• run() - Execute workflows from configuration

Utility Functions (perform operations, return None):

• configure_qualcomm_sdk() - Configure Qualcomm SDK
• convert_adapters() - Convert adapter formats
• extract_adapters() - Extract LoRA adapters
• generate_cost_model() - Generate cost models for splitting
• manage_aml_compute() - Manage AzureML compute resources
• shared_cache() - Manage shared cache operations

Usage Examples

from olive import auto_opt, finetune, extract_adapters

# Auto-optimize a model
result = auto_opt(
    model_path="microsoft/phi-3-mini-4k-instruct",
    device="cpu", 
    precision="int8"
)

# Access the best optimized model
if result.has_output_model():
    best_model = result.get_best_candidate()
    print(f"Model: {best_model.model_path}")
    print(f"Metrics: {best_model.metrics_value}")

# Fine-tune with LoRA
adapter = finetune(
    model_path="microsoft/phi-3-mini-4k-instruct",
    data_name="squad",
    method="lora",
    num_train_epochs=3
)

# Extract adapters (utility function)
extract_adapters(
    model_path="./model_with_lora",
    output_path="./extracted",
    format="onnx_adapter"
)

Key Benefits

• Same functionality as CLI but with programmatic access
• Structured returns - WorkflowOutput objects for optimization workflows
• Python-native parameters instead of command-line arguments
• Proper error handling and directory management
• Full workflow traceability through ModelOutput instances

Files Added

• olive/api/__init__.py - API module exports
• olive/api/workflow.py - All API function implementations
• olive/api/README.md - Comprehensive documentation
• examples/python_api_usage.py - Usage examples
• test/unit_test/test_python_api.py - Unit tests

Files Modified

• olive/__init__.py - Export API functions from main module

The API maintains full compatibility with existing CLI functionality while providing a more convenient interface for programmatic use.

Fixes #1906.

[devang-ml]

We don't need python APIs for following CLIs.

shared_cache,
manage_aml_compute,
configure_qualcomm_sdk

[Copilot]

We don't need python APIs for following CLIs.

shared_cache,
...

Removed the Python APIs for shared_cache, manage_aml_compute, and configure_qualcomm_sdk as requested. Updated all documentation, tests, and examples to reflect these changes. Commit: 99c5a40

[devang-ml]

Let's add documentation on how to use these APIs.

docs/source/how-to/index.rst documents how to use CLI and how to use .json configuration. Let's update this page to cover ho to use python APIs.

[Copilot]

Let's add documentation on how to use these APIs.

docs/source/how-to/index.rst documents how to use CLI and how to use .json configuration. Let's update this page to cover ho to use python APIs.

Added comprehensive Python API documentation to the docs/source/how-to/index.rst page. Created a new "Working with the Python API" section with detailed documentation for auto_opt(), finetune(), quantize(), run(), and an overview page with usage patterns and examples. The documentation follows the same structure as the CLI documentation with function signatures, parameters, return values, and integration examples. Commit: f7051f7

[xiaoyu-work]

The Python API should be in one single documentation page with different sections.

[github-advanced-security]

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[xiaoyu-work]

Remove this file, combine this content to documentation you added.

[xiaoyu-work]

Remove this file

[xiaoyu-work]

Fix lint issue.

[xiaoyu-work]

The Python API should be in one single documentation page with different sections.

[xiaoyu-work]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[xiaoyu-work]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[xiaoyu-work]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).