mixvideo-v2/cargos/tvai

TVAI - Topaz Video AI Integration Library

A Rust library for integrating with Topaz Video AI to perform video and image enhancement including super-resolution upscaling and frame interpolation.

Features

• 🎬 Video Super-Resolution: Upscale videos using AI models
• 🎞️ Frame Interpolation: Create smooth slow motion effects
• 🖼️ Image Upscaling: Enhance image resolution and quality
• ⚡ GPU Acceleration: CUDA and hardware encoding support
• 🔧 Multiple AI Models: 16 upscaling and 4 interpolation models
• 📦 Batch Processing: Process multiple files efficiently
• 🎛️ Flexible Configuration: Fine-tune processing parameters
• 📋 Template System: Built-in templates from Topaz Video AI examples
• 🚀 Easy API: Simple functions for common video processing tasks
• ⚡ FFmpeg Integration: Generate FFmpeg commands from templates

Requirements

• Topaz Video AI installed
• Rust 1.70+
• FFmpeg (included with Topaz Video AI)
• Optional: CUDA-compatible GPU for acceleration

Installation

Add this to your Cargo.toml:

[dependencies]
tvai = "0.1.0"

Quick Start

Using Built-in Templates (Recommended)

use tvai::*;
use std::path::Path;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Upscale to 4K using built-in template
    upscale_to_4k(
        Path::new("input.mp4"),
        Path::new("output_4k.mp4")
    ).await?;

    // Convert to 60fps
    convert_to_60fps(
        Path::new("input.mp4"),
        Path::new("output_60fps.mp4")
    ).await?;

    // Remove noise
    remove_noise(
        Path::new("noisy_video.mp4"),
        Path::new("clean_video.mp4")
    ).await?;

    // Create slow motion
    slow_motion_4x(
        Path::new("action_video.mp4"),
        Path::new("slow_motion.mp4")
    ).await?;

    Ok(())
}

Using Any Template by Name

use tvai::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Process with any built-in template
    process_with_template(
        Path::new("input.mp4"),
        Path::new("output.mp4"),
        "film_stock_4k_strong"
    ).await?;

    Ok(())
}

Traditional API (Still Available)

use tvai::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Quick 2x upscaling
    quick_upscale_video(
        std::path::Path::new("input.mp4"),
        std::path::Path::new("output.mp4"),
        2.0,
    ).await?;

    Ok(())
}

Image Upscaling

use tvai::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Quick 4x image upscaling
    quick_upscale_image(
        std::path::Path::new("photo.jpg"),
        std::path::Path::new("photo_4x.png"),
        4.0,
    ).await?;
    
    Ok(())
}

Advanced Usage

use tvai::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Detect Topaz installation
    let topaz_path = detect_topaz_installation()
        .ok_or("Topaz Video AI not found")?;
    
    // Create configuration
    let config = TvaiConfig::builder()
        .topaz_path(topaz_path)
        .use_gpu(true)
        .build()?;
    
    // Create processor
    let processor = TvaiProcessor::new(config)?;
    
    // Custom upscaling parameters
    let params = VideoUpscaleParams {
        scale_factor: 2.0,
        model: UpscaleModel::Iris3,
        compression: 0.0,
        blend: 0.1,
        quality_preset: QualityPreset::HighQuality,
    };
    
    // Process video
    let result = processor.upscale_video(
        std::path::Path::new("input.mp4"),
        std::path::Path::new("output.mp4"),
        params,
    ).await?;
    
    println!("Processing completed in {:?}", result.processing_time);
    Ok(())
}

AI Models

Upscaling Models

• Iris v3 - Best general purpose model
• Nyx v3 - Optimized for portraits
• Theia Fidelity v4 - Old content restoration
• Gaia HQ v5 - Game/CG content
• Proteus v4 - Problem footage repair
• And more...

Interpolation Models

• Apollo v8 - High quality interpolation
• Chronos v2 - Animation content
• Apollo Fast v1 - Fast processing
• Chronos Fast v3 - Fast animation

Presets

The library includes optimized presets for common use cases:

// Video presets
let old_video_params = VideoUpscaleParams::for_old_video();
let game_params = VideoUpscaleParams::for_game_content();
let animation_params = VideoUpscaleParams::for_animation();
let portrait_params = VideoUpscaleParams::for_portrait();

// Image presets
let photo_params = ImageUpscaleParams::for_photo();
let artwork_params = ImageUpscaleParams::for_artwork();
let screenshot_params = ImageUpscaleParams::for_screenshot();

System Detection

// Detect Topaz installation
let topaz_path = detect_topaz_installation();

// Check GPU support
let gpu_info = detect_gpu_support();

// Check FFmpeg availability
let ffmpeg_info = detect_ffmpeg();

Template System

The library includes a comprehensive template system with built-in templates from Topaz Video AI examples:

Available Templates

• Upscaling: upscale_to_4k, upscale_to_fhd, upscale_4k_convert_60fps
• Frame Rate: convert_to_60fps, 4x_slow_motion, 8x_super_slow_motion
• Enhancement: remove_noise, film_stock_4k_light/medium/strong
• Restoration: deinterlace_upscale_fhd, minidv_hd_int_basic, auto_crop_stabilization

Template Management

use tvai::*;

// List all available templates
let templates = list_available_templates();
for template_name in templates {
    if let Some((name, description)) = get_template_info(&template_name) {
        println!("{}: {}", name, description);
    }
}

// Load custom templates
let manager = global_topaz_templates().lock().unwrap();
manager.load_from_file("my_template".to_string(), "path/to/template.json")?;
manager.load_examples_templates("templates_directory/")?;

Custom Templates

You can create and load custom templates in Topaz Video AI JSON format. See Template Documentation for details.

FFmpeg Command Generation

Generate FFmpeg commands from templates for external use:

use tvai::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let manager = global_topaz_templates().lock().unwrap();

    // Generate basic command
    let cmd = manager.quick_ffmpeg_command("upscale_to_4k", "input.mp4", "output.mp4")?;
    println!("{}", cmd);

    // GPU-specific commands
    let nvidia_cmd = manager.gpu_ffmpeg_command("convert_to_60fps", "input.mp4", "output.mp4", "nvidia")?;
    let amd_cmd = manager.gpu_ffmpeg_command("remove_noise", "input.mp4", "output.mp4", "amd")?;

    // Custom quality
    let hq_cmd = manager.quality_ffmpeg_command("film_stock_4k_strong", "input.mp4", "output.mp4", 15, "slow")?;

    Ok(())
}

Error Handling

The library uses comprehensive error handling with user-friendly messages:

use tvai::*;

match upscale_to_4k(input, output).await {
    Ok(result) => println!("Success: {:?}", result),
    Err(TvaiError::TopazNotFound(path)) => {
        eprintln!("Topaz not found at: {}", path);
    },
    Err(TvaiError::FfmpegError(msg)) => {
        eprintln!("FFmpeg error: {}", msg);
    },
    Err(e) => {
        eprintln!("Error: {}", e);
        eprintln!("Suggestion: {}", e.user_friendly_message());
    },
}

Development Status

✅ COMPLETE - All core features implemented and tested!

• Basic project structure
• FFmpeg management
• Core processor framework
• Video upscaling implementation (16 AI models)
• Frame interpolation implementation (4 AI models)
• Image upscaling implementation
• Batch processing (videos and images)
• Progress callbacks and monitoring
• Global configuration management
• Preset management system
• Template system with built-in Topaz Video AI templates
• Convenient API functions for common tasks
• Performance optimization
• Enhanced error handling
• Comprehensive testing (unit + integration + benchmarks)
• Complete documentation (API + User Guide + Templates)

Documentation

English

• 📖 API Documentation - Complete API reference
• 📚 User Guide - Comprehensive usage guide
• 📋 Template Documentation - Template system guide

中文文档

• 📖 API 文档 - 完整 API 参考
• 📚 用户指南 - 详细使用指南
• 📋 中文 README - 中文版说明文档

Resources

• 🔧 Examples - Working code examples
• 🧪 Tests - Integration tests
• 📊 Benchmarks - Performance benchmarks

Performance

The library is optimized for performance with:

• GPU Acceleration - CUDA and hardware encoding support
• Concurrent Processing - Configurable parallel operations
• Memory Management - Efficient temporary file handling
• Smart Caching - Intelligent resource utilization
• Progress Monitoring - Real-time performance tracking

Run benchmarks with:

cargo bench

Testing

Comprehensive test suite including:

• Unit Tests - Core functionality testing
• Integration Tests - End-to-end workflow testing
• Benchmark Tests - Performance validation

Run tests with:

cargo test
cargo test --release  # For performance tests

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Setup

1. Install Rust 1.70+
2. Install Topaz Video AI
3. Clone the repository
4. Run tests: cargo test
5. Run examples: cargo run --example basic_usage

Changelog

v0.1.0 (Current)

• ✅ Complete video processing (upscaling + interpolation)
• ✅ Complete image processing (upscaling + batch operations)
• ✅ 16 AI upscaling models + 4 interpolation models
• ✅ Global configuration and preset management
• ✅ Performance monitoring and optimization
• ✅ Enhanced error handling with user-friendly messages
• ✅ Comprehensive documentation and examples
• ✅ Full test coverage (unit + integration + benchmarks)