mixvideo-v2/cargos/tvai-v2

Topaz Video AI Rust SDK

A comprehensive Rust SDK for managing Topaz Video AI templates and generating FFmpeg commands with AI-powered video processing filters.

Features

• 🎯 Template Management: Load, validate, and manage Topaz Video AI templates
• 🔧 FFmpeg Command Generation: Convert templates to optimized FFmpeg commands
• 🚀 Hardware Acceleration: Support for NVIDIA, AMD, and Intel hardware acceleration
• 📦 Batch Processing: Generate commands for multiple files at once
• 🛠️ Template Builder: Programmatically create custom templates
• 📋 Presets: Built-in templates for common video processing tasks
• 💾 Database Support: Interface for database storage and runtime loading
• ✅ Validation: Comprehensive template validation and auto-completion

Quick Start

Add this to your Cargo.toml:

[dependencies]
tvai-sdk = "0.1.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

Basic Usage

use tvai_sdk::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create SDK instance
    let mut sdk = TvaiSdk::new();
    
    // Create a template using presets
    let template = TemplatePresets::upscale_to_4k()?;
    
    // Generate FFmpeg command
    let command = sdk.generate_ffmpeg_command(
        &template, 
        "input.mp4", 
        "output_4k.mp4"
    )?;
    
    println!("FFmpeg command: {}", command);
    Ok(())
}

Load Templates from Directory

let mut sdk = TvaiSdk::new();

// Load all JSON templates from directory
sdk.load_templates_from_dir("./templates")?;

// List loaded templates
for name in sdk.get_template_names() {
    println!("Loaded template: {}", name);
}

Create Custom Templates

let custom_template = TemplateBuilder::new("My Custom Template")
    .description("Custom video enhancement")
    .enable_enhancement("prob-4")
    .enhancement_params(30, 20, 10) // denoise, detail, sharpen
    .enable_stabilization(60, 1) // smoothness, method
    .resolution(1920, 1080) // Custom resolution
    .video_codec("hevc_nvenc", Some(20)) // H.265 NVENC with CRF 20
    .preset("slow") // Encoding preset
    .audio_settings("aac", 192, 2) // AAC 192kbps stereo
    .hardware_acceleration("cuda", Some("0")) // GPU acceleration
    .build()?;

sdk.add_template(custom_template)?;

Hardware Acceleration

// Generate command with GPU acceleration
let gpu_command = sdk.generate_ffmpeg_command_with_gpu(
    &template,
    "input.mp4",
    "output.mp4",
    "0" // GPU device 0
)?;

// Generate command with custom codec
let nvenc_command = sdk.generate_ffmpeg_command_with_codec(
    &template,
    "input.mp4",
    "output.mp4",
    "hevc_nvenc",
    Some(20) // quality
)?;

Batch Processing

let input_files = vec![
    "video1.mp4".to_string(),
    "video2.mp4".to_string(),
    "video3.mp4".to_string(),
];

let commands = sdk.generate_batch_commands(
    &template,
    &input_files,
    "./output"
)?;

for command in commands {
    println!("Batch command: {}", command);
}

Template Structure

Templates are JSON files with the following structure:

{
    "name": "Template Name",
    "description": "Template description",
    "author": "Author Name",
    "date": "Creation date",
    "veaiversion": "5.1.2",
    "editable": true,
    "enabled": true,
    "saveOutputSettings": false,
    "settings": {
        "stabilize": { ... },
        "motionblur": { ... },
        "slowmo": { ... },
        "enhance": { ... },
        "grain": { ... },
        "output": { ... }
    }
}

Supported Processing Modules

Video Enhancement (enhance)

• Models: prob-3, prob-4, nyx-3, etc.
• Parameters: denoise, detail, sharpen, compress, dehalo, deblur
• FFmpeg Filter: tvai_up

Frame Interpolation (slowmo)

• Models: apo-8, chf-3, chr-2, etc.
• Parameters: factor, fps, duplicate threshold
• FFmpeg Filter: tvai_fi

Video Stabilization (stabilize)

• Parameters: smoothness, method, rolling shutter correction
• FFmpeg Filter: tvai_stb

Grain Effect (grain)

• Parameters: grain amount, grain size
• FFmpeg Filter: noise (approximation)

Built-in Presets

The SDK includes several built-in template presets:

Basic Presets

• TemplatePresets::upscale_to_4k() - Upscale video to 4K resolution
• TemplatePresets::convert_to_60fps() - Convert video to 60 FPS
• TemplatePresets::remove_noise() - Remove noise using AI
• TemplatePresets::stabilize_video() - Stabilize shaky video
• TemplatePresets::slow_motion_4x() - Create 4x slow motion effect
• TemplatePresets::comprehensive_enhancement() - Complete enhancement

Advanced Presets

• TemplatePresets::high_quality_nvenc() - High-quality NVIDIA NVENC encoding
• TemplatePresets::hdr_processing() - HDR content processing with proper color settings
• TemplatePresets::streaming_optimized() - Optimized for live streaming
• TemplatePresets::mobile_optimized() - Small file sizes for mobile devices
• TemplatePresets::archival_quality() - Maximum quality for archival purposes

Extended Output Settings

The SDK supports comprehensive FFmpeg output configuration:

Video Settings

• Custom Resolution: Set exact width and height
• Video Codecs: H.264, H.265, AV1, and hardware-accelerated variants
• Quality Control: CRF (Constant Rate Factor) or bitrate-based encoding
• Encoding Presets: ultrafast, fast, medium, slow, veryslow
• Profiles & Levels: baseline, main, high, main10, etc.
• Advanced Parameters: GOP size, B-frames, pixel format

Audio Settings

• Audio Codecs: AAC, MP3, FLAC, Opus, etc.
• Quality: Bitrate, sample rate, channel configuration
• Lossless Support: FLAC and ALAC with proper validation

Color & HDR Support

• Color Spaces: bt709, bt2020nc, etc.
• Color Primaries: bt709, bt2020, etc.
• Transfer Characteristics: bt709, smpte2084 (HDR10), etc.
• Pixel Formats: yuv420p, yuv420p10le (10-bit), etc.

Hardware Acceleration

• Methods: CUDA, OpenCL, QSV, AMF
• GPU Selection: Specify device index for multi-GPU systems
• Codec-Specific: NVENC, AMF, QSV variants

Example: Advanced Template

let advanced_template = TemplateBuilder::new("Professional 4K HDR")
    .description("Professional 4K HDR encoding")
    .enable_enhancement("prob-4")
    .resolution(3840, 2160) // 4K
    .video_codec("hevc_nvenc", Some(16)) // High quality H.265
    .preset("slow") // Best compression
    .profile_level("main10", Some("5.1")) // 10-bit profile
    .pixel_format("yuv420p10le") // 10-bit pixel format
    .color_settings("bt2020nc", "bt2020", "smpte2084") // HDR10
    .audio_settings("aac", 256, 2) // High quality audio
    .hardware_acceleration("cuda", Some("0")) // GPU 0
    .custom_params(vec![
        "-color_range tv".to_string(),
        "-max_muxing_queue_size 1024".to_string(),
    ])
    .build()?;

Hardware Acceleration Support

NVIDIA (NVENC/NVDEC)

// Single GPU
let command = sdk.generate_ffmpeg_command_with_gpu(&template, input, output, "0")?;

// Multi-GPU
let command = sdk.generate_ffmpeg_command_with_gpu(&template, input, output, "0.1")?;

// NVENC codec
let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "hevc_nvenc", Some(20))?;

AMD (AMF)

let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "h264_amf", Some(25))?;

Intel (QSV)

let command = sdk.generate_ffmpeg_command_with_codec(&template, input, output, "h264_qsv", Some(23))?;

Database Integration

Implement the TemplateDatabase trait for custom database storage:

use tvai_sdk::TemplateDatabase;

struct MyDatabase {
    // Your database connection
}

impl TemplateDatabase for MyDatabase {
    fn save_template(&self, template: &Template) -> TvaiResult<()> {
        // Save template to database
        Ok(())
    }
    
    fn load_template(&self, name: &str) -> TvaiResult<Template> {
        // Load template from database
        todo!()
    }
    
    // Implement other methods...
}

// Use with DatabaseTemplateManager
let db = MyDatabase::new();
let mut manager = DatabaseTemplateManager::new(db);
manager.load_from_database()?;

Examples

Run the included examples:

# Basic usage
cargo run --example basic_usage

# Template management
cargo run --example template_management

# FFmpeg command generation
cargo run --example ffmpeg_generation

# Advanced output settings
cargo run --example advanced_output_settings

Testing

Run the test suite:

cargo test

Model Mappings

The SDK maps Topaz Video AI template models to FFmpeg filter models:

Template Model	FFmpeg Model	Filter
prob-3, prob-4	ahq-12	tvai_up
nyx-3	nyx-3	tvai_up
apo-8, chf-3	chr-2	tvai_fi
thm-2	thm-2	tvai_up

You can add custom mappings:

sdk.add_model_mapping("custom-model".to_string(), "ffmpeg-model".to_string());

Error Handling

The SDK uses a comprehensive error system:

use tvai_sdk::TvaiError;

match sdk.load_template_from_file("template.json") {
    Ok(template) => println!("Loaded: {}", template.name),
    Err(TvaiError::IoError(e)) => println!("File error: {}", e),
    Err(TvaiError::JsonParseError(e)) => println!("JSON error: {}", e),
    Err(TvaiError::ValidationError(msg)) => println!("Validation error: {}", msg),
    Err(e) => println!("Other error: {}", e),
}

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.