391 lines
10 KiB
Markdown
391 lines
10 KiB
Markdown
# TVAI - Topaz Video AI Integration Library
|
|
|
|
A Rust library for integrating with Topaz Video AI to perform video and image enhancement including super-resolution upscaling and frame interpolation.
|
|
|
|
## Features
|
|
|
|
- 🎬 **Video Super-Resolution**: Upscale videos using AI models
|
|
- 🎞️ **Frame Interpolation**: Create smooth slow motion effects
|
|
- 🖼️ **Image Upscaling**: Enhance image resolution and quality
|
|
- ⚡ **GPU Acceleration**: CUDA and hardware encoding support
|
|
- 🔧 **Multiple AI Models**: 16 upscaling and 4 interpolation models
|
|
- 📦 **Batch Processing**: Process multiple files efficiently
|
|
- 🎛️ **Flexible Configuration**: Fine-tune processing parameters
|
|
- 📋 **Template System**: Built-in templates from Topaz Video AI examples
|
|
- 🚀 **Easy API**: Simple functions for common video processing tasks
|
|
- ⚡ **FFmpeg Integration**: Generate FFmpeg commands from templates
|
|
|
|
## Requirements
|
|
|
|
- [Topaz Video AI](https://www.topazlabs.com/topaz-video-ai) installed
|
|
- Rust 1.70+
|
|
- FFmpeg (included with Topaz Video AI)
|
|
- Optional: CUDA-compatible GPU for acceleration
|
|
|
|
## Installation
|
|
|
|
Add this to your `Cargo.toml`:
|
|
|
|
```toml
|
|
[dependencies]
|
|
tvai = "0.1.0"
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### Using Built-in Templates (Recommended)
|
|
|
|
```rust
|
|
use tvai::*;
|
|
use std::path::Path;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Upscale to 4K using built-in template
|
|
upscale_to_4k(
|
|
Path::new("input.mp4"),
|
|
Path::new("output_4k.mp4")
|
|
).await?;
|
|
|
|
// Convert to 60fps
|
|
convert_to_60fps(
|
|
Path::new("input.mp4"),
|
|
Path::new("output_60fps.mp4")
|
|
).await?;
|
|
|
|
// Remove noise
|
|
remove_noise(
|
|
Path::new("noisy_video.mp4"),
|
|
Path::new("clean_video.mp4")
|
|
).await?;
|
|
|
|
// Create slow motion
|
|
slow_motion_4x(
|
|
Path::new("action_video.mp4"),
|
|
Path::new("slow_motion.mp4")
|
|
).await?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Using Any Template by Name
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Process with any built-in template
|
|
process_with_template(
|
|
Path::new("input.mp4"),
|
|
Path::new("output.mp4"),
|
|
"film_stock_4k_strong"
|
|
).await?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Traditional API (Still Available)
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Quick 2x upscaling
|
|
quick_upscale_video(
|
|
std::path::Path::new("input.mp4"),
|
|
std::path::Path::new("output.mp4"),
|
|
2.0,
|
|
).await?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Image Upscaling
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Quick 4x image upscaling
|
|
quick_upscale_image(
|
|
std::path::Path::new("photo.jpg"),
|
|
std::path::Path::new("photo_4x.png"),
|
|
4.0,
|
|
).await?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Advanced Usage
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Detect Topaz installation
|
|
let topaz_path = detect_topaz_installation()
|
|
.ok_or("Topaz Video AI not found")?;
|
|
|
|
// Create configuration
|
|
let config = TvaiConfig::builder()
|
|
.topaz_path(topaz_path)
|
|
.use_gpu(true)
|
|
.build()?;
|
|
|
|
// Create processor
|
|
let processor = TvaiProcessor::new(config)?;
|
|
|
|
// Custom upscaling parameters
|
|
let params = VideoUpscaleParams {
|
|
scale_factor: 2.0,
|
|
model: UpscaleModel::Iris3,
|
|
compression: 0.0,
|
|
blend: 0.1,
|
|
quality_preset: QualityPreset::HighQuality,
|
|
};
|
|
|
|
// Process video
|
|
let result = processor.upscale_video(
|
|
std::path::Path::new("input.mp4"),
|
|
std::path::Path::new("output.mp4"),
|
|
params,
|
|
).await?;
|
|
|
|
println!("Processing completed in {:?}", result.processing_time);
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
## AI Models
|
|
|
|
### Upscaling Models
|
|
|
|
- **Iris v3** - Best general purpose model
|
|
- **Nyx v3** - Optimized for portraits
|
|
- **Theia Fidelity v4** - Old content restoration
|
|
- **Gaia HQ v5** - Game/CG content
|
|
- **Proteus v4** - Problem footage repair
|
|
- And more...
|
|
|
|
### Interpolation Models
|
|
|
|
- **Apollo v8** - High quality interpolation
|
|
- **Chronos v2** - Animation content
|
|
- **Apollo Fast v1** - Fast processing
|
|
- **Chronos Fast v3** - Fast animation
|
|
|
|
## Presets
|
|
|
|
The library includes optimized presets for common use cases:
|
|
|
|
```rust
|
|
// Video presets
|
|
let old_video_params = VideoUpscaleParams::for_old_video();
|
|
let game_params = VideoUpscaleParams::for_game_content();
|
|
let animation_params = VideoUpscaleParams::for_animation();
|
|
let portrait_params = VideoUpscaleParams::for_portrait();
|
|
|
|
// Image presets
|
|
let photo_params = ImageUpscaleParams::for_photo();
|
|
let artwork_params = ImageUpscaleParams::for_artwork();
|
|
let screenshot_params = ImageUpscaleParams::for_screenshot();
|
|
```
|
|
|
|
## System Detection
|
|
|
|
```rust
|
|
// Detect Topaz installation
|
|
let topaz_path = detect_topaz_installation();
|
|
|
|
// Check GPU support
|
|
let gpu_info = detect_gpu_support();
|
|
|
|
// Check FFmpeg availability
|
|
let ffmpeg_info = detect_ffmpeg();
|
|
```
|
|
|
|
## Template System
|
|
|
|
The library includes a comprehensive template system with built-in templates from Topaz Video AI examples:
|
|
|
|
### Available Templates
|
|
|
|
- **Upscaling**: `upscale_to_4k`, `upscale_to_fhd`, `upscale_4k_convert_60fps`
|
|
- **Frame Rate**: `convert_to_60fps`, `4x_slow_motion`, `8x_super_slow_motion`
|
|
- **Enhancement**: `remove_noise`, `film_stock_4k_light/medium/strong`
|
|
- **Restoration**: `deinterlace_upscale_fhd`, `minidv_hd_int_basic`, `auto_crop_stabilization`
|
|
|
|
### Template Management
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
// List all available templates
|
|
let templates = list_available_templates();
|
|
for template_name in templates {
|
|
if let Some((name, description)) = get_template_info(&template_name) {
|
|
println!("{}: {}", name, description);
|
|
}
|
|
}
|
|
|
|
// Load custom templates
|
|
let manager = global_topaz_templates().lock().unwrap();
|
|
manager.load_from_file("my_template".to_string(), "path/to/template.json")?;
|
|
manager.load_examples_templates("templates_directory/")?;
|
|
```
|
|
|
|
### Custom Templates
|
|
|
|
You can create and load custom templates in Topaz Video AI JSON format. See [Template Documentation](docs/TEMPLATES.md) for details.
|
|
|
|
### FFmpeg Command Generation
|
|
|
|
Generate FFmpeg commands from templates for external use:
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
let manager = global_topaz_templates().lock().unwrap();
|
|
|
|
// Generate basic command
|
|
let cmd = manager.quick_ffmpeg_command("upscale_to_4k", "input.mp4", "output.mp4")?;
|
|
println!("{}", cmd);
|
|
|
|
// GPU-specific commands
|
|
let nvidia_cmd = manager.gpu_ffmpeg_command("convert_to_60fps", "input.mp4", "output.mp4", "nvidia")?;
|
|
let amd_cmd = manager.gpu_ffmpeg_command("remove_noise", "input.mp4", "output.mp4", "amd")?;
|
|
|
|
// Custom quality
|
|
let hq_cmd = manager.quality_ffmpeg_command("film_stock_4k_strong", "input.mp4", "output.mp4", 15, "slow")?;
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The library uses comprehensive error handling with user-friendly messages:
|
|
|
|
```rust
|
|
use tvai::*;
|
|
|
|
match upscale_to_4k(input, output).await {
|
|
Ok(result) => println!("Success: {:?}", result),
|
|
Err(TvaiError::TopazNotFound(path)) => {
|
|
eprintln!("Topaz not found at: {}", path);
|
|
},
|
|
Err(TvaiError::FfmpegError(msg)) => {
|
|
eprintln!("FFmpeg error: {}", msg);
|
|
},
|
|
Err(e) => {
|
|
eprintln!("Error: {}", e);
|
|
eprintln!("Suggestion: {}", e.user_friendly_message());
|
|
},
|
|
}
|
|
```
|
|
|
|
## Development Status
|
|
|
|
✅ **COMPLETE** - All core features implemented and tested!
|
|
|
|
- [x] Basic project structure
|
|
- [x] FFmpeg management
|
|
- [x] Core processor framework
|
|
- [x] Video upscaling implementation (16 AI models)
|
|
- [x] Frame interpolation implementation (4 AI models)
|
|
- [x] Image upscaling implementation
|
|
- [x] Batch processing (videos and images)
|
|
- [x] Progress callbacks and monitoring
|
|
- [x] Global configuration management
|
|
- [x] Preset management system
|
|
- [x] Template system with built-in Topaz Video AI templates
|
|
- [x] Convenient API functions for common tasks
|
|
- [x] Performance optimization
|
|
- [x] Enhanced error handling
|
|
- [x] Comprehensive testing (unit + integration + benchmarks)
|
|
- [x] Complete documentation (API + User Guide + Templates)
|
|
|
|
## Documentation
|
|
|
|
### English
|
|
- 📖 [API Documentation](docs/API.md) - Complete API reference
|
|
- 📚 [User Guide](docs/USER_GUIDE.md) - Comprehensive usage guide
|
|
- 📋 [Template Documentation](docs/TEMPLATES.md) - Template system guide
|
|
|
|
### 中文文档
|
|
- 📖 [API 文档](docs/API文档.md) - 完整 API 参考
|
|
- 📚 [用户指南](docs/用户指南.md) - 详细使用指南
|
|
- 📋 [中文 README](README_CN.md) - 中文版说明文档
|
|
|
|
### Resources
|
|
- 🔧 [Examples](examples/) - Working code examples
|
|
- 🧪 [Tests](tests/) - Integration tests
|
|
- 📊 [Benchmarks](benches/) - Performance benchmarks
|
|
|
|
## Performance
|
|
|
|
The library is optimized for performance with:
|
|
|
|
- **GPU Acceleration** - CUDA and hardware encoding support
|
|
- **Concurrent Processing** - Configurable parallel operations
|
|
- **Memory Management** - Efficient temporary file handling
|
|
- **Smart Caching** - Intelligent resource utilization
|
|
- **Progress Monitoring** - Real-time performance tracking
|
|
|
|
Run benchmarks with:
|
|
```bash
|
|
cargo bench
|
|
```
|
|
|
|
## Testing
|
|
|
|
Comprehensive test suite including:
|
|
|
|
- **Unit Tests** - Core functionality testing
|
|
- **Integration Tests** - End-to-end workflow testing
|
|
- **Benchmark Tests** - Performance validation
|
|
|
|
Run tests with:
|
|
```bash
|
|
cargo test
|
|
cargo test --release # For performance tests
|
|
```
|
|
|
|
## License
|
|
|
|
MIT License - see LICENSE file for details.
|
|
|
|
## Contributing
|
|
|
|
Contributions are welcome! Please feel free to submit a Pull Request.
|
|
|
|
### Development Setup
|
|
|
|
1. Install Rust 1.70+
|
|
2. Install Topaz Video AI
|
|
3. Clone the repository
|
|
4. Run tests: `cargo test`
|
|
5. Run examples: `cargo run --example basic_usage`
|
|
|
|
## Changelog
|
|
|
|
### v0.1.0 (Current)
|
|
|
|
- ✅ Complete video processing (upscaling + interpolation)
|
|
- ✅ Complete image processing (upscaling + batch operations)
|
|
- ✅ 16 AI upscaling models + 4 interpolation models
|
|
- ✅ Global configuration and preset management
|
|
- ✅ Performance monitoring and optimization
|
|
- ✅ Enhanced error handling with user-friendly messages
|
|
- ✅ Comprehensive documentation and examples
|
|
- ✅ Full test coverage (unit + integration + benchmarks)
|