java Import video file (.mp4)

Question

I need to import a video file (.mp4) and store it in a Bufferedimage[], so I can work with each frame as if it were an image. my code(the part with (??? ) is where I did not agree):

public class Main {

public static void main(String[] args) {
    videoChromaKey(new File("./mascara.mp4"),new File("./fundo.mp4"),new Color(0,255,0),100);
}

public static BufferedImage chromaKey(BufferedImage bimg,BufferedImage bbg,Color key,int sensibility){
    for(int i=0;i<bimg.getWidth();i++){
        for(int j=0;j<bimg.getHeight();j++){
            Color e=new Color(bimg.getRGB(i, j));
            Color bgc=new Color(bbg.getRGB(i, j));
            if(e.getGreen()-e.getRed()>=key.getGreen()-key.getRed()-sensibility && e.getGreen()-e.getRed()<=key.getGreen()-key.getRed()+sensibility &&
               e.getGreen()-e.getBlue()>=key.getGreen()-key.getBlue()-sensibility && e.getGreen()-e.getBlue()<=key.getGreen()-key.getBlue()+sensibility
            )
            bimg.setRGB(i, j, bgc.getRGB());
        }
    }
    return bimg;
    }
}

public static void videoChromaKey(File in,File bg,File out,Color key,int sensibility){
    BufferedImage[] vin= (???);
    BufferedImage[] vbg= (???);
    for(int i=0;i<vin.length && i<vbg.length;i++){
        vin[i]=chromaKey(vin[i],vbg[i],key,sensibility);
    }
    //Gravar vin em arquivo out
    (???)
}

}

Answer 1

You can use the Marvin Framework to do this, it has already been made to work with media frame by frame.

While running you will get images/frames in type MarvinImage, but you can convert to bytes or ByteArray afterward.

Example (taken from their own website):

public class MediaFileExample implements Runnable{

    private MarvinVideoInterface    videoAdapter;
    private MarvinImage             videoFrame;

    public MediaFileExample(){
        try{
            // Cria o VideoAdapter usado para carregar o arquivo de vídeo
            videoAdapter = new MarvinJavaCVAdapter();
            videoAdapter.loadResource("./res/snooker.wmv");

            // Inicia a thread para pegar todos os frames
            new Thread(this).start();
        }
        catch(MarvinVideoInterfaceException e){e.printStackTrace();}
    }

    @Override
    public void run() {
        try{
            while(true){
                // Aqui que a mágica acontece, esse método já te da o frame pronto
                videoFrame = videoAdapter.getFrame();
            }
        }catch(MarvinVideoInterfaceException e){e.printStackTrace();}
    }

    public static void main(String[] args) {
        MediaFileExample m = new MediaFileExample();
    }
}