依托HarmonyOS 6.1最新特性实现图片编辑APP（八）：图片接收与相机实时处理

前言

在移动应用中，相机实时处理是一个极具吸引力的功能——用户在拍照时就能看到滤镜效果、美颜特效或实时增强。HarmonyOS 6.1 Image Kit 通过 ImageReceiver 组件提供了接收实时图像流的能力，结合相机的预览流数据，开发者可以构建实时图像处理管道。

ImageReceiver 是Image Kit中用于接收实时图像流数据的组件，常用于相机预览、视频帧处理和屏幕录制等场景。它提供了高效的内存管理和灵活的缓冲区配置。

一、ImageReceiver基础概念

1.1 ImageReceiver工作原理

ImageReceiver 作为图像数据的消费者，接收来自相机、解码器等生产者的图像数据：

组件	角色	说明
相机预览	生产者	持续输出相机帧数据
ImageReceiver	消费者	接收并管理图像缓冲区
PixelMap	处理对象	将接收到的帧转为可处理的PixelMap

1.2 核心配置参数

参数	说明	典型值
width	接收图像宽度	1920
height	接收图像高度	1080
format	像素格式	NV12/IMAGE_FORMAT_RGBA_8888
capacity	缓冲区最大数量	8
usage	内存使用方式	HW_MEMORY

二、ImageReceiver创建与配置

2.1 基础ImageReceiver配置

import { image } from '@kit.ImageKit';
import { camera } from '@kit.CameraKit';
import { BusinessError } from '@kit.BasicServicesKit';

// ImageReceiver管理工具
class ImageReceiverManager {
  private receiver: image.ImageReceiver | undefined;
  private width: number;
  private height: number;
  private format: number;
  private capacity: number;
  
  constructor(width: number = 1920, height: number = 1080) {
    this.width = width;
    this.height = height;
    this.format = 4; // IMAGE_FORMAT_RGBA_8888
    this.capacity = 8;
  }
  
  // 创建ImageReceiver
  createReceiver(): image.ImageReceiver {
    try {
      this.receiver = image.createImageReceiver(
        this.width,
        this.height,
        this.format,
        this.capacity
      );
      console.info(`ImageReceiver created: ${this.width}x${this.height}, capacity=${this.capacity}`);
      return this.receiver;
    } catch (error) {
      console.error(`Failed to create ImageReceiver: ${error}`);
      throw error;
    }
  }
  
  // 获取Receiver的SurfaceId（用于相机绑定）
  getReceivingSurfaceId(): string {
    if (!this.receiver) {
      throw new Error('ImageReceiver not created.');
    }
    return this.receiver.getReceivingSurfaceId();
  }
  
  // 获取下一个图像帧
  async getNextImage(): Promise<image.Image | undefined> {
    if (!this.receiver) return undefined;
    
    try {
      const img = await this.receiver.readNextImage();
      console.info('Image frame received.');
      return img;
    } catch (error) {
      console.error(`Failed to read next image: ${error}`);
      return undefined;
    }
  }
  
  // 获取最新图像帧
  async getLatestImage(): Promise<image.Image | undefined> {
    if (!this.receiver) return undefined;
    
    try {
      const img = await this.receiver.readLatestImage();
      console.info('Latest image frame received.');
      return img;
    } catch (error) {
      console.error(`Failed to read latest image: ${error}`);
      return undefined;
    }
  }
  
  // 释放ImageReceiver
  release(): void {
    if (this.receiver) {
      this.receiver.release();
      console.info('ImageReceiver released.');
      this.receiver = undefined;
    }
  }
}

2.2 图像帧处理

// 图像帧处理器
class ImageFrameProcessor {
  // 从Image对象获取PixelMap组件
  static async getComponentFromImage(img: image.Image): Promise<image.PixelMap | undefined> {
    try {
      const component = await img.getComponent(4); // 4 = component type
      console.info('Image component obtained.');
      return component;
    } catch (error) {
      console.error(`Failed to get image component: ${error}`);
      return undefined;
    }
  }
  
  // 处理图像帧（应用实时滤镜）
  static async processFrame(
    img: image.Image,
    filterChain: FilterChainManager
  ): Promise<image.PixelMap | undefined> {
    try {
      // 获取PixelMap组件
      const pixelMap = await ImageFrameProcessor.getComponentFromImage(img);
      if (!pixelMap) return undefined;
      
      // 读取像素数据
      const buffer = await pixelMap.readPixelsToBuffer();
      if (!buffer) return undefined;
      
      // 转换为颜色数组
      let colors = RGBAPixelOperator.bufferToRGBAColors(buffer);
      
      // 获取图片尺寸
      let info = await pixelMap.getImageInfo();
      
      // 应用滤镜链
      colors = filterChain.applyFilterChain(colors, info.size.width, info.size.height);
      
      // 写回像素数据
      const resultBuffer = RGBAPixelOperator.colorsToBuffer(colors);
      await pixelMap.writeBufferToPixels(resultBuffer);
      
      return pixelMap;
    } catch (error) {
      console.error(`Frame processing failed: ${error}`);
      return undefined;
    } finally {
      // 释放Image对象
      img.release();
    }
  }
  
  // 批量处理帧（用于视频处理）
  static async processFrameBatch(
    images: image.Image[],
    filterChain: FilterChainManager
  ): Promise<image.PixelMap[]> {
    const results: image.PixelMap[] = [];
    
    for (const img of images) {
      const result = await ImageFrameProcessor.processFrame(img, filterChain);
      if (result) {
        results.push(result);
      }
    }
    
    return results;
  }
}

三、相机集成与实时滤镜

3.1 相机预览管理器

// 相机预览管理器
class CameraPreviewManager {
  private receiverManager: ImageReceiverManager;
  private isPreviewing: boolean = false;
  private frameProcessor: ImageFrameProcessor;
  private filterChain: FilterChainManager;
  
  constructor() {
    this.receiverManager = new ImageReceiverManager(1920, 1080);
    this.frameProcessor = new ImageFrameProcessor();
    this.filterChain = new FilterChainManager();
  }
  
  // 启动相机预览
  async startPreview(): Promise<void> {
    try {
      // 创建ImageReceiver
      const receiver = this.receiverManager.createReceiver();
      const surfaceId = this.receiverManager.getReceivingSurfaceId();
      
      console.info(`Preview surfaceId: ${surfaceId}`);
      
      // 将surfaceId绑定到相机
      // 实际项目中需要调用Camera API
      // await cameraManager.createPreviewOutput(surfaceId);
      
      this.isPreviewing = true;
      console.info('Camera preview started.');
    } catch (error) {
      console.error(`Failed to start preview: ${error}`);
    }
  }
  
  // 处理预览帧
  async processPreviewFrame(): Promise<image.PixelMap | undefined> {
    if (!this.isPreviewing) return undefined;
    
    const img = await this.receiverManager.getLatestImage();
    if (!img) return undefined;
    
    return ImageFrameProcessor.processFrame(img, this.filterChain);
  }
  
  // 添加预览滤镜
  addPreviewFilter(name: string, params: Record<string, number> = {}): void {
    this.filterChain.addFilter({
      name: name,
      params: params,
      enabled: true
    });
  }
  
  // 移除预览滤镜
  removePreviewFilter(index: number): void {
    this.filterChain.removeFilter(index);
  }
  
  // 停止预览
  stopPreview(): void {
    this.isPreviewing = false;
    this.receiverManager.release();
    console.info('Camera preview stopped.');
  }
}

3.2 实时滤镜页面

// 实时滤镜相机页面
@Entry
@Component
struct CameraFilterPage {
  @State previewPixelMap: image.PixelMap | undefined = undefined;
  @State isCameraActive: boolean = false;
  @State currentFilter: string = 'none';
  @State selectedFilter: string = 'none';
  private previewManager: CameraPreviewManager = new CameraPreviewManager();
  private frameTimer: number = -1;
  
  // 启动相机
  async startCamera(): Promise<void> {
    this.isCameraActive = true;
    await this.previewManager.startPreview();
    
    // 启动帧处理循环
    this.startFrameProcessing();
  }
  
  // 帧处理循环
  startFrameProcessing(): void {
    const processFrame = async () => {
      if (!this.isCameraActive) return;
      
      const result = await this.previewManager.processPreviewFrame();
      if (result) {
        this.previewPixelMap = result;
      }
      
      // 继续下一帧处理
      this.frameTimer = setTimeout(processFrame, 33); // ~30fps
    };
    
    processFrame();
  }
  
  // 应用滤镜
  applyFilter(filterName: string): void {
    this.previewManager.removePreviewFilter(0);
    
    if (filterName !== 'none') {
      this.previewManager.addPreviewFilter(filterName);
    }
    
    this.currentFilter = filterName;
  }
  
  // 停止相机
  stopCamera(): void {
    this.isCameraActive = false;
    clearTimeout(this.frameTimer);
    this.previewManager.stopPreview();
    
    if (this.previewPixelMap) {
      this.previewPixelMap.release();
      this.previewPixelMap = undefined;
    }
  }
  
  build() {
    Column() {
      // 相机预览区域
      Stack() {
        if (this.previewPixelMap) {
          Image(this.previewPixelMap)
            .width('100%')
            .height('100%')
            .objectFit(ImageFit.Cover)
        } else {
          Column() {
            Text('相机预览区域')
              .fontSize(18)
              .fontColor('#999999')
          }
          .width('100%')
          .height('100%')
          .justifyContent(FlexAlign.Center)
          .backgroundColor('#F0F0F0')
        }
        
        // 滤镜名称显示
        if (this.currentFilter !== 'none') {
          Text(this.currentFilter)
            .fontSize(14)
            .fontColor(Color.White)
            .padding(8)
            .backgroundColor('rgba(0,0,0,0.5)')
            .borderRadius(4)
            .position({ x: 20, y: 20 })
        }
      }
      .layoutWeight(1)
      
      // 滤镜选择栏
      Scroll() {
        Row() {
          ForEach(['none', 'vintage', 'cool', 'warm', 'monochrome', 'dramatic', 'soft'], (filter: string) => {
            Column() {
              Text(filter === 'none' ? '原图' : filter)
                .fontSize(11)
                .padding(8)
                .backgroundColor(this.currentFilter === filter ? '#007AFF' : 'rgba(255,255,255,0.8)')
                .fontColor(this.currentFilter === filter ? Color.White : '#333333')
                .borderRadius(8)
            }
            .onClick(() => {
              this.applyFilter(filter);
            })
            .margin(5)
          })
        }
        .padding(10)
      }
      .height(70)
      .scrollable(ScrollDirection.Horizontal)
      .backgroundColor('rgba(0,0,0,0.8)')
      
      // 相机控制按钮
      Row() {
        Button('拍照')
          .fontSize(18)
          .width(70)
          .height(70)
          .borderRadius(35)
          .backgroundColor(Color.White)
          .onClick(() => {
            console.info('Photo captured with filter: ' + this.currentFilter);
          })
      }
      .width('100%')
      .justifyContent(FlexAlign.Center)
      .padding(20)
      .backgroundColor('rgba(0,0,0,0.9)')
    }
    .width('100%')
    .height('100%')
  }
  
  aboutToAppear(): void {
    this.startCamera();
  }
  
  aboutToDisappear(): void {
    this.stopCamera();
  }
}

四、高性能图像流处理

4.1 帧率控制

// 帧率控制器
class FrameRateController {
  private targetFps: number;
  private frameInterval: number;
  private lastFrameTime: number = 0;
  private frameCount: number = 0;
  private fpsTimer: number = -1;
  private currentFps: number = 0;
  
  constructor(targetFps: number = 30) {
    this.targetFps = targetFps;
    this.frameInterval = 1000 / targetFps;
  }
  
  // 检查是否应该处理当前帧
  shouldProcessFrame(): boolean {
    const now = Date.now();
    if (now - this.lastFrameTime >= this.frameInterval) {
      this.lastFrameTime = now;
      this.frameCount++;
      return true;
    }
    return false;
  }
  
  // 开始FPS统计
  startFPSMonitor(): void {
    this.frameCount = 0;
    this.fpsTimer = setInterval(() => {
      this.currentFps = this.frameCount;
      this.frameCount = 0;
      console.info(`Current FPS: ${this.currentFps}`);
    }, 1000);
  }
  
  // 停止FPS统计
  stopFPSMonitor(): void {
    clearInterval(this.fpsTimer);
  }
  
  // 获取当前FPS
  getCurrentFps(): number {
    return this.currentFps;
  }
  
  // 动态调整帧率（根据处理负载）
  adjustFps(processingTime: number): void {
    if (processingTime > this.frameInterval) {
      this.targetFps = Math.max(15, this.targetFps - 5);
      this.frameInterval = 1000 / this.targetFps;
      console.info(`FPS adjusted to: ${this.targetFps}`);
    } else if (processingTime < this.frameInterval * 0.5 && this.targetFps < 30) {
      this.targetFps = Math.min(30, this.targetFps + 5);
      this.frameInterval = 1000 / this.targetFps;
      console.info(`FPS adjusted to: ${this.targetFps}`);
    }
  }
}

4.2 内存管理优化

// 图像流内存管理器
class StreamMemoryManager {
  private maxBuffers: number = 8;
  private activeBuffers: Set<image.Image> = new Set();
  private memoryWarningThreshold: number = 0.8; // 80%内存使用阈值
  
  // 注册图像缓冲区
  registerBuffer(img: image.Image): void {
    this.activeBuffers.add(img);
    
    if (this.activeBuffers.size > this.maxBuffers) {
      this.cleanupOldestBuffer();
    }
  }
  
  // 释放缓冲区
  releaseBuffer(img: image.Image): void {
    img.release();
    this.activeBuffers.delete(img);
  }
  
  // 清理最旧的缓冲区
  private cleanupOldestBuffer(): void {
    const iterator = this.activeBuffers.values();
    const oldest = iterator.next().value;
    if (oldest) {
      this.releaseBuffer(oldest);
      console.info('Oldest buffer released for memory management.');
    }
  }
  
  // 清理所有缓冲区
  cleanupAll(): void {
    for (const img of this.activeBuffers) {
      img.release();
    }
    this.activeBuffers.clear();
    console.info('All buffers released.');
  }
  
  // 内存压力处理
  handleMemoryPressure(level: number): void {
    if (level >= 2) {
      // 高内存压力：释放一半缓冲区
      const count = Math.floor(this.activeBuffers.size / 2);
      let i = 0;
      for (const img of this.activeBuffers) {
        if (i >= count) break;
        this.releaseBuffer(img);
        i++;
      }
    }
    console.info(`Memory pressure handled. Level: ${level}`);
  }
}

五、实时处理性能对比

5.1 不同处理模式的性能

处理模式	FPS	延迟	内存占用	适用场景
无滤镜	30	<10ms	低	基础预览
简单滤镜	25-30	~15ms	低	亮度/对比度
中等滤镜	20-25	~25ms	中	模糊/复古
复杂滤镜	15-20	~40ms	中	多滤镜组合
AI增强	10-15	~80ms	高	超分辨率

5.2 优化建议

// 实时处理优化建议
class RealTimeOptimizationTips {
  static readonly tips = [
    {
      title: '使用NV12格式',
      description: 'NV12格式比RGBA_8888节省37.5%的内存带宽，适合相机预览流'
    },
    {
      title: '降低处理分辨率',
      description: '预览时使用较低分辨率（如720p），拍照时再使用全分辨率'
    },
    {
      title: '限制滤镜复杂度',
      description: '实时预览时使用轻量级滤镜，保存时再应用完整滤镜链'
    },
    {
      title: '使用DMA内存',
      description: '减少CPU-GPU之间的数据拷贝，提升处理效率'
    },
    {
      title: '帧丢弃策略',
      description: '当处理时间超过帧间隔时，丢弃过期帧保证实时性'
    }
  ];
}

总结

本文介绍了 HarmonyOS 6.1 Image Kit 中 ImageReceiver 的使用方法，包括图像流接收、相机预览集成、实时滤镜处理和性能优化。通过 ImageReceiver 与相机的结合，ImageEditor Pro 实现了实时滤镜预览功能，让用户在拍照时就能看到最终的滤镜效果。

下一篇文章，我们将介绍 HDR与多图对象高级处理，探索高品质图像处理的前沿技术。

如果这篇文章对你有帮助，欢迎点赞👍、收藏⭐、关注🔔，你的支持是我持续创作的动力！

---

相关资源：

• 使用ImageReceiver完成图片接收：官方文档
• 图片接收开发指南：图片接收
• 图片开发指导(ArkTS)：开发指南
• 图片编辑和处理：编辑处理指南
• PixelMap接口：Interface (PixelMap)
• Image组件：Image组件
• Image Kit常见问题：FAQ
• 开源鸿蒙跨平台社区：社区链接