357 lines
9.5 KiB
Markdown
357 lines
9.5 KiB
Markdown
# Topaz Video AI Rust SDK
|
|
|
|
A comprehensive Rust SDK for managing Topaz Video AI templates and generating FFmpeg commands with AI-powered video processing filters.
|
|
|
|
## Features
|
|
|
|
- 🎯 **Template Management**: Load, validate, and manage Topaz Video AI templates
|
|
- 🔧 **FFmpeg Command Generation**: Convert templates to optimized FFmpeg commands
|
|
- 🚀 **Hardware Acceleration**: Support for NVIDIA, AMD, and Intel hardware acceleration
|
|
- 📦 **Batch Processing**: Generate commands for multiple files at once
|
|
- 🛠️ **Template Builder**: Programmatically create custom templates
|
|
- 📋 **Presets**: Built-in templates for common video processing tasks
|
|
- 💾 **Database Support**: Interface for database storage and runtime loading
|
|
- ✅ **Validation**: Comprehensive template validation and auto-completion
|
|
|
|
## Quick Start
|
|
|
|
Add this to your `Cargo.toml`:
|
|
|
|
```toml
|
|
[dependencies]
|
|
tvai-sdk = "0.1.0"
|
|
serde = { version = "1.0", features = ["derive"] }
|
|
serde_json = "1.0"
|
|
```
|
|
|
|
### Basic Usage
|
|
|
|
```rust
|
|
use tvai_sdk::*;
|
|
|
|
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Create SDK instance
|
|
let mut sdk = TvaiSdk::new();
|
|
|
|
// Create a template using presets
|
|
let template = TemplatePresets::upscale_to_4k()?;
|
|
|
|
// Generate FFmpeg command
|
|
let command = sdk.generate_ffmpeg_command(
|
|
&template,
|
|
"input.mp4",
|
|
"output_4k.mp4"
|
|
)?;
|
|
|
|
println!("FFmpeg command: {}", command);
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Load Templates from Directory
|
|
|
|
```rust
|
|
let mut sdk = TvaiSdk::new();
|
|
|
|
// Load all JSON templates from directory
|
|
sdk.load_templates_from_dir("./templates")?;
|
|
|
|
// List loaded templates
|
|
for name in sdk.get_template_names() {
|
|
println!("Loaded template: {}", name);
|
|
}
|
|
```
|
|
|
|
### Create Custom Templates
|
|
|
|
```rust
|
|
let custom_template = TemplateBuilder::new("My Custom Template")
|
|
.description("Custom video enhancement")
|
|
.enable_enhancement("prob-4")
|
|
.enhancement_params(30, 20, 10) // denoise, detail, sharpen
|
|
.enable_stabilization(60, 1) // smoothness, method
|
|
.resolution(1920, 1080) // Custom resolution
|
|
.video_codec("hevc_nvenc", Some(20)) // H.265 NVENC with CRF 20
|
|
.preset("slow") // Encoding preset
|
|
.audio_settings("aac", 192, 2) // AAC 192kbps stereo
|
|
.hardware_acceleration("cuda", Some("0")) // GPU acceleration
|
|
.build()?;
|
|
|
|
sdk.add_template(custom_template)?;
|
|
```
|
|
|
|
### Hardware Acceleration
|
|
|
|
```rust
|
|
// Generate command with GPU acceleration
|
|
let gpu_command = sdk.generate_ffmpeg_command_with_gpu(
|
|
&template,
|
|
"input.mp4",
|
|
"output.mp4",
|
|
"0" // GPU device 0
|
|
)?;
|
|
|
|
// Generate command with custom codec
|
|
let nvenc_command = sdk.generate_ffmpeg_command_with_codec(
|
|
&template,
|
|
"input.mp4",
|
|
"output.mp4",
|
|
"hevc_nvenc",
|
|
Some(20) // quality
|
|
)?;
|
|
```
|
|
|
|
### Batch Processing
|
|
|
|
```rust
|
|
let input_files = vec![
|
|
"video1.mp4".to_string(),
|
|
"video2.mp4".to_string(),
|
|
"video3.mp4".to_string(),
|
|
];
|
|
|
|
let commands = sdk.generate_batch_commands(
|
|
&template,
|
|
&input_files,
|
|
"./output"
|
|
)?;
|
|
|
|
for command in commands {
|
|
println!("Batch command: {}", command);
|
|
}
|
|
```
|
|
|
|
## Template Structure
|
|
|
|
Templates are JSON files with the following structure:
|
|
|
|
```json
|
|
{
|
|
"name": "Template Name",
|
|
"description": "Template description",
|
|
"author": "Author Name",
|
|
"date": "Creation date",
|
|
"veaiversion": "5.1.2",
|
|
"editable": true,
|
|
"enabled": true,
|
|
"saveOutputSettings": false,
|
|
"settings": {
|
|
"stabilize": { ... },
|
|
"motionblur": { ... },
|
|
"slowmo": { ... },
|
|
"enhance": { ... },
|
|
"grain": { ... },
|
|
"output": { ... }
|
|
}
|
|
}
|
|
```
|
|
|
|
## Supported Processing Modules
|
|
|
|
### Video Enhancement (`enhance`)
|
|
- **Models**: prob-3, prob-4, nyx-3, etc.
|
|
- **Parameters**: denoise, detail, sharpen, compress, dehalo, deblur
|
|
- **FFmpeg Filter**: `tvai_up`
|
|
|
|
### Frame Interpolation (`slowmo`)
|
|
- **Models**: apo-8, chf-3, chr-2, etc.
|
|
- **Parameters**: factor, fps, duplicate threshold
|
|
- **FFmpeg Filter**: `tvai_fi`
|
|
|
|
### Video Stabilization (`stabilize`)
|
|
- **Parameters**: smoothness, method, rolling shutter correction
|
|
- **FFmpeg Filter**: `tvai_stb`
|
|
|
|
### Grain Effect (`grain`)
|
|
- **Parameters**: grain amount, grain size
|
|
- **FFmpeg Filter**: `noise` (approximation)
|
|
|
|
## Built-in Presets
|
|
|
|
The SDK includes several built-in template presets:
|
|
|
|
### Basic Presets
|
|
- `TemplatePresets::upscale_to_4k()` - Upscale video to 4K resolution
|
|
- `TemplatePresets::convert_to_60fps()` - Convert video to 60 FPS
|
|
- `TemplatePresets::remove_noise()` - Remove noise using AI
|
|
- `TemplatePresets::stabilize_video()` - Stabilize shaky video
|
|
- `TemplatePresets::slow_motion_4x()` - Create 4x slow motion effect
|
|
- `TemplatePresets::comprehensive_enhancement()` - Complete enhancement
|
|
|
|
### Advanced Presets
|
|
- `TemplatePresets::high_quality_nvenc()` - High-quality NVIDIA NVENC encoding
|
|
- `TemplatePresets::hdr_processing()` - HDR content processing with proper color settings
|
|
- `TemplatePresets::streaming_optimized()` - Optimized for live streaming
|
|
- `TemplatePresets::mobile_optimized()` - Small file sizes for mobile devices
|
|
- `TemplatePresets::archival_quality()` - Maximum quality for archival purposes
|
|
|
|
## Extended Output Settings
|
|
|
|
The SDK supports comprehensive FFmpeg output configuration:
|
|
|
|
### Video Settings
|
|
- **Custom Resolution**: Set exact width and height
|
|
- **Video Codecs**: H.264, H.265, AV1, and hardware-accelerated variants
|
|
- **Quality Control**: CRF (Constant Rate Factor) or bitrate-based encoding
|
|
- **Encoding Presets**: ultrafast, fast, medium, slow, veryslow
|
|
- **Profiles & Levels**: baseline, main, high, main10, etc.
|
|
- **Advanced Parameters**: GOP size, B-frames, pixel format
|
|
|
|
### Audio Settings
|
|
- **Audio Codecs**: AAC, MP3, FLAC, Opus, etc.
|
|
- **Quality**: Bitrate, sample rate, channel configuration
|
|
- **Lossless Support**: FLAC and ALAC with proper validation
|
|
|
|
### Color & HDR Support
|
|
- **Color Spaces**: bt709, bt2020nc, etc.
|
|
- **Color Primaries**: bt709, bt2020, etc.
|
|
- **Transfer Characteristics**: bt709, smpte2084 (HDR10), etc.
|
|
- **Pixel Formats**: yuv420p, yuv420p10le (10-bit), etc.
|
|
|
|
### Hardware Acceleration
|
|
- **Methods**: CUDA, OpenCL, QSV, AMF
|
|
- **GPU Selection**: Specify device index for multi-GPU systems
|
|
- **Codec-Specific**: NVENC, AMF, QSV variants
|
|
|
|
### Example: Advanced Template
|
|
```rust
|
|
let advanced_template = TemplateBuilder::new("Professional 4K HDR")
|
|
.description("Professional 4K HDR encoding")
|
|
.enable_enhancement("prob-4")
|
|
.resolution(3840, 2160) // 4K
|
|
.video_codec("hevc_nvenc", Some(16)) // High quality H.265
|
|
.preset("slow") // Best compression
|
|
.profile_level("main10", Some("5.1")) // 10-bit profile
|
|
.pixel_format("yuv420p10le") // 10-bit pixel format
|
|
.color_settings("bt2020nc", "bt2020", "smpte2084") // HDR10
|
|
.audio_settings("aac", 256, 2) // High quality audio
|
|
.hardware_acceleration("cuda", Some("0")) // GPU 0
|
|
.custom_params(vec![
|
|
"-color_range tv".to_string(),
|
|
"-max_muxing_queue_size 1024".to_string(),
|
|
])
|
|
.build()?;
|
|
```
|
|
|
|
## Hardware Acceleration Support
|
|
|
|
### NVIDIA (NVENC/NVDEC)
|
|
```rust
|
|
// Single GPU
|
|
let command = sdk.generate_ffmpeg_command_with_gpu(&template, input, output, "0")?;
|
|
|
|
// Multi-GPU
|
|
let command = sdk.generate_ffmpeg_command_with_gpu(&template, input, output, "0.1")?;
|
|
|
|
// NVENC codec
|
|
let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "hevc_nvenc", Some(20))?;
|
|
```
|
|
|
|
### AMD (AMF)
|
|
```rust
|
|
let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "h264_amf", Some(25))?;
|
|
```
|
|
|
|
### Intel (QSV)
|
|
```rust
|
|
let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "h264_qsv", Some(23))?;
|
|
```
|
|
|
|
## Database Integration
|
|
|
|
Implement the `TemplateDatabase` trait for custom database storage:
|
|
|
|
```rust
|
|
use tvai_sdk::TemplateDatabase;
|
|
|
|
struct MyDatabase {
|
|
// Your database connection
|
|
}
|
|
|
|
impl TemplateDatabase for MyDatabase {
|
|
fn save_template(&self, template: &Template) -> TvaiResult<()> {
|
|
// Save template to database
|
|
Ok(())
|
|
}
|
|
|
|
fn load_template(&self, name: &str) -> TvaiResult<Template> {
|
|
// Load template from database
|
|
todo!()
|
|
}
|
|
|
|
// Implement other methods...
|
|
}
|
|
|
|
// Use with DatabaseTemplateManager
|
|
let db = MyDatabase::new();
|
|
let mut manager = DatabaseTemplateManager::new(db);
|
|
manager.load_from_database()?;
|
|
```
|
|
|
|
## Examples
|
|
|
|
Run the included examples:
|
|
|
|
```bash
|
|
# Basic usage
|
|
cargo run --example basic_usage
|
|
|
|
# Template management
|
|
cargo run --example template_management
|
|
|
|
# FFmpeg command generation
|
|
cargo run --example ffmpeg_generation
|
|
|
|
# Advanced output settings
|
|
cargo run --example advanced_output_settings
|
|
```
|
|
|
|
## Testing
|
|
|
|
Run the test suite:
|
|
|
|
```bash
|
|
cargo test
|
|
```
|
|
|
|
## Model Mappings
|
|
|
|
The SDK maps Topaz Video AI template models to FFmpeg filter models:
|
|
|
|
| Template Model | FFmpeg Model | Filter |
|
|
|----------------|--------------|--------|
|
|
| prob-3, prob-4 | ahq-12 | tvai_up |
|
|
| nyx-3 | nyx-3 | tvai_up |
|
|
| apo-8, chf-3 | chr-2 | tvai_fi |
|
|
| thm-2 | thm-2 | tvai_up |
|
|
|
|
You can add custom mappings:
|
|
|
|
```rust
|
|
sdk.add_model_mapping("custom-model".to_string(), "ffmpeg-model".to_string());
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The SDK uses a comprehensive error system:
|
|
|
|
```rust
|
|
use tvai_sdk::TvaiError;
|
|
|
|
match sdk.load_template_from_file("template.json") {
|
|
Ok(template) => println!("Loaded: {}", template.name),
|
|
Err(TvaiError::IoError(e)) => println!("File error: {}", e),
|
|
Err(TvaiError::JsonParseError(e)) => println!("JSON error: {}", e),
|
|
Err(TvaiError::ValidationError(msg)) => println!("Validation error: {}", msg),
|
|
Err(e) => println!("Other error: {}", e),
|
|
}
|
|
```
|
|
|
|
## License
|
|
|
|
MIT License - see LICENSE file for details.
|
|
|
|
## Contributing
|
|
|
|
Contributions are welcome! Please feel free to submit a Pull Request.
|