java buffer size to copy data on write device file

[openmpi]

Hello. I want to copy data from input.txt to xillybus_write_8. In order to improve the performance of the application, I set the buffers size (bufPut and bufGet) to 1000000 (int mb). The problem of setting 1000000, is that the java application gets stuck at outPut.write(bufPut, 0, lenPut). The upper limit of the buffer size is 32768 (int mb = 32768). Is there any reason why the upper limit is 32768? Is it related to Xillybus?

Code: Select all

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class UpperLowCaseSwapFPGA {
   
   public static void main(String [] args) throws IOException {
      long startTime = System.nanoTime();
 
      File f1Put=null; //input file
      File f2Put=null; //write device file file
      InputStream inPut=null; //input stream for input file
      OutputStream outPut=null; //output stream for write device file
      File f1Get=null; //read device file
      File f2Get=null; //output file
      InputStream inGet=null; //input stream for read device file
      OutputStream outGet=null; //output stream for output device file

      try {
         int mb = 1000000; //buffer size
         
                  //Initialize Put
         f1Put = new File("/my_path/input.txt"); //input file
         f2Put = new File("/dev/xillybus_write_8");
         inPut = new FileInputStream(f1Put);
         outPut = new FileOutputStream(f2Put, true);         
         byte[] bufPut = new byte[mb]; //buffer for copying from input file to write device file
         int lenPut; //number of bytes written from input file to write device file
         
         //Initialize Get
         f1Get = new File("/dev/xillybus_read_8");
                  f2Get = new File("/my_path/output.txt"); //output file
         inGet = new FileInputStream(f1Get);
         outGet = new FileOutputStream(f2Get, true);
         byte[] bufGet = new byte[mb]; //buffer for copying from read device file to output file
         int lenGet; //number of bytes written from read device file to output file
         
         while ((lenPut = inPut.read(bufPut)) != -1 ) { //reads up bufPut.length bytes of data from inPut (input file) into bufPut
            
            //Write on xillybus_write_8      
            outPut.write(bufPut, 0, lenPut); //write from bufPut (input device file) to write device file
            outPut.flush();

            //Read from xillybus_read_8
            lenGet = inGet.read(bufGet); //reads up bufGet.length bytes of data from inGet (read device file) into bufGet
            outGet.write(bufGet, 0, lenGet); //write from bufGet (read device file) to output file
            outGet.flush();   
         }
         long estimatedTime = System.nanoTime() - startTime;
         System.out.println("Total execution time (ms): " + (estimatedTime/1000000));      
      }
      catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in this directory.");
            System.exit(0);
      }
      catch (IOException e) {
            System.out.println(e.getMessage());
      }
      finally {
         inPut.close();
         outPut.close();
         inGet.close();
         outGet.close();
      }
      
   }
}


[support]

Hello,

As I mentioned in a previous post (viewtopic.php?f=4&t=353), there must be two different threads (or some other non-blocking equivalent) if data goes in both directions. Otherwise, weird things tend to happen. It's quite difficult to try figuring out what exactly happened that got the program stuck. Except that read operations and write operations shouldn't wait for each other.

So if it worked for you for a certain value of mb, you were lucky. And less lucky with another value. There are unfortunately no shortcuts if you want a program that is stable.

Regards,
Eli

[openmpi]

Ok I understand. I created two threads, one for copying the input file into write device file and the other one for copying data from read device file to output file. It also works with buffer size 1 MB but ReadThread never finishes because it seems like readDeviceFileStream never reaches the end of the stream. More precisely, once all data are copied from read device file to output file, the program gets stuck on outputFileStream.write(bufGet, 0, lenGet); (ReadThread class). I have tested this application with local files on my pc and it works. Does Xillybus keep alive read device file input stream?


Here the main class

Code: Select all

import java.io.*;

public class UpperLowCaseSwapFPGAThread {
   
   public static void main(String [] args) {

      long startTime = System.nanoTime();
      
      String inputPath = args[0]; //input file path
      String outputPath = "./output.txt"; //output file path

      WriteThread writer = new WriteThread(inputPath);
      ReadThread reader = new ReadThread(outputPath);
      
      new Thread(writer).start();
      new Thread(reader).start();
      
      long estimatedTime = System.nanoTime() - startTime;
      System.out.println("Total execution time (ms): " + (estimatedTime/1000000));      
                  
   }
}


The following code copies data from input file to write device file

Code: Select all

import java.io.*;

//from input file to write device file

public class WriteThread implements Runnable {

   File inputFile=null; //input file
         
   public WriteThread(String inputPath) {
      this.inputFile = new File(inputPath);
   }

   @Override
   public void run() {
      
      int mb = 1048576; //buffer size
      
      try {
         File writeDeviceFile = new File("/dev/xillybus_write_8"); //write device file
         InputStream inputFileStream = new FileInputStream(inputFile); //input stream for input file
         OutputStream writeDeviceFileStream = new FileOutputStream(writeDeviceFile, true); //output stream for write device file      
         byte[] bufPut = new byte[mb]; //buffer for copying from input file to write device file
         int lenPut = 0;
         
         //Write on xillybus_write_8
         while ((lenPut = inputFileStream.read(bufPut)) != -1 ) {//reads up bufPut.length bytes of data from inputFileStream (input file) into bufPut    
            writeDeviceFileStream.write(bufPut, 0, lenPut); //write from bufPut (input device file) to write device file
            writeDeviceFileStream.flush();
         }
         inputFileStream.close();
         writeDeviceFileStream.close();
      }
      catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in this directory.");
            System.exit(0);
      }
      catch (IOException e) {
            System.out.println(e.getMessage());
      }
   }
}


The following code copy data from read device file to output file:

Code: Select all

import java.io.*;

public class ReadThread implements Runnable{
   
   File outputFile=null; //output file
   
   public ReadThread(String outputPath){
      this.outputFile = new File(outputPath);
   }

   @Override
   public void run() {
      
      int mb = 1048576; //buffer size
            
      try {
         File readDeviceFile = new File("/dev/xillybus_read_8"); //read device file
         InputStream readDeviceFileStream = new FileInputStream(readDeviceFile); //input stream for read device file
         OutputStream outputFileStream = new FileOutputStream(outputFile, true); //output stream for output file
         byte[] bufGet = new byte[mb]; //buffer for copying from read device file to output file
         int lenGet = 0;
                  
         while((lenGet = readDeviceFileStream.read(bufGet)) != -1) {//reads up bufGet.length bytes of data from readDeviceFileStream (read device file) into bufGet
            outputFileStream.write(bufGet, 0, lenGet); //write from bufGet (read device file) to output file
            outputFileStream.flush();
         }         

         readDeviceFileStream.close();
         outputFileStream.close();
      }
      catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in this directory.");
            System.exit(0);
      }
      catch (IOException e) {
            System.out.println(e.getMessage());
      }
   }
}


[support]

Hello,

I'm glad that you got the previous issue sorted out with two threads. I forgot to mention, that it could be two independent programs as well, but two threads is more elegant.

I don't know how you tell where the program gets stuck (debugger?), but I would put my money on readDeviceFileStream.read(bufGet).

The thing is like this: Once the write thread finishes, it closes the file, but this has no significance on the FPGA level, except that the user_w_write_8_open signal goes low.

The read thread, on the other hand, keeps reading on until it reaches an EOF. But that never happens, because the user_r_read_8_eof is held constantly low, I suppose. You could solve this in two possible ways:

(1) The quick hack: Kill the read thread a certain time period (500 ms?) after the write thread has finished, assuming that this time will be enough for the read thread to handle the incoming data. Ugly, can fail if the computer gets heavily loaded at the wrong moment, but sometimes good enough.

(2) The proper way: Issue an EOF. Note that *_eof signals must be asserted only on a clock cycle following a read cycle (the Xillybus FPGA Designer's Guide doc explains this), but there is an exception on the case when "empty" was already asserted. So you could go something like this in the logic, i.e the xillydemo module (example in Verilog):

assign user_r_read_8_eof = user_r_read_8_empty && !user_w_write_8_open;

If you go this way, you must make sure to open the write device file before the read file, or you'll get the EOF right away.

(3) The common variable: Count how much data you've written on the write thread, and count how much has arrived on the read thread. Have a common variable, say limit, and initialize it to -1. Before the write thread writes its last chunk of data, make it set limit to the total number of bytes. The read thread checks, before any attempt to read, if limit >= 0, and if so, if it has reached the total number of bytes. If so, it quits. Note that "limit" must be updated before the last write, and not after it. It's convenient to update it just before quitting the write thread, but that's not good enough: There is a possibility that the read thread will have read the data in the little time gap, so "limit" would be updated too late.

I would go for (2).

Regards,
Eli

[openmpi]

Great! I went for (2). The writing thread copies data from input file to write device file. Once data is on write device file, my VHDL code swaps lower case with uppper case and vice versa of text stored on write device file and it outputs the swaped characters on read device file. Finally, the reading thread copies data from read device file to output file. Before swapping, the FPGA does the EOF check as you suggested me.
Now, something new happens. Since I set the same buffer size for reading and for writing threads (int mb), I checked that both threads cicled (whithin the while loop) the same number of times before closing the streams but they did not cicle the same number of times.
E.g., by setting the buffer size to 1048576 Bytes (1 MB) in order to swap upper and lower case of an input file with size 10 MB, the writing thread (that copies data from input file to write device file) cicles 10 times (this is expected since the buffer size is 1MB and the input file size 10MB, thus 1*10=10 cicles) but the reading thread (that copies data from read device file to output file) cicles 12 times. 12 cicles sound weird for me, because the reading thread was supposed to cicle 10 times for the same reason of writing thread. Any idea why reading thread cicles more times than writing thread? Thanks.

Here the VHDL code:

Code: Select all

library ieee;
use ieee.std_logic_1164.all;
use ieee.std_logic_unsigned.all;
use ieee.numeric_std.all;

entity xillydemo is
  port (
     PCIE_PERST_B_LS : IN std_logic;
     PCIE_REFCLK_N : IN std_logic;
     PCIE_REFCLK_P : IN std_logic;
     PCIE_RX_N : IN std_logic_vector(3 DOWNTO 0);
     PCIE_RX_P : IN std_logic_vector(3 DOWNTO 0);
     GPIO_LED : OUT std_logic_vector(3 DOWNTO 0);
     PCIE_TX_N : OUT std_logic_vector(3 DOWNTO 0);
     PCIE_TX_P : OUT std_logic_vector(3 DOWNTO 0));
end xillydemo;

architecture sample_arch of xillydemo is
    signal tmp :  std_logic_vector(7 DOWNTO 0);
 

  component xillybus
    port (
      PCIE_PERST_B_LS : IN std_logic;
      PCIE_REFCLK_N : IN std_logic;
      PCIE_REFCLK_P : IN std_logic;
      PCIE_RX_N : IN std_logic_vector(3 DOWNTO 0);
      PCIE_RX_P : IN std_logic_vector(3 DOWNTO 0);
      GPIO_LED : OUT std_logic_vector(3 DOWNTO 0);
      PCIE_TX_N : OUT std_logic_vector(3 DOWNTO 0);
      PCIE_TX_P : OUT std_logic_vector(3 DOWNTO 0);
      bus_clk : OUT std_logic;
      quiesce : OUT std_logic;
     
      user_r_read_8_rden : OUT std_logic;
      user_r_read_8_empty : IN std_logic;
      user_r_read_8_data : IN std_logic_vector(7 DOWNTO 0);
      user_r_read_8_eof : IN std_logic;
      user_r_read_8_open : OUT std_logic;
      user_w_write_8_wren : OUT std_logic;
      user_w_write_8_full : IN std_logic;
      user_w_write_8_data : OUT std_logic_vector(7 DOWNTO 0);
      user_w_write_8_open : OUT std_logic);
  end component;

  component fifo_8x2048
    port (
      clk: IN std_logic;
      srst: IN std_logic;
      din: IN std_logic_VECTOR(7 downto 0);
      wr_en: IN std_logic;
      rd_en: IN std_logic;
      dout: OUT std_logic_VECTOR(7 downto 0);
      full: OUT std_logic;
      empty: OUT std_logic);
  end component;


-- Synplicity black box declaration
  attribute syn_black_box : boolean;
  attribute syn_black_box of fifo_8x2048: component is true;
 
  signal bus_clk :  std_logic;
  signal quiesce : std_logic;

  signal reset_8 : std_logic;

  signal ram_addr : integer range 0 to 31;
 
  signal user_r_read_8_rden  :  std_logic;
  signal user_r_read_8_empty :  std_logic;
  signal user_r_read_8_data  :  std_logic_vector(7 DOWNTO 0);
  signal user_r_read_8_eof   :  std_logic;
  signal user_r_read_8_open  :  std_logic;
  signal user_w_write_8_wren :  std_logic;
  signal user_w_write_8_full :  std_logic;
  signal user_w_write_8_data :  std_logic_vector(7 DOWNTO 0);
  signal user_w_write_8_open :  std_logic;
  signal wr_en               :  std_logic := '0';
  signal din                 :  std_logic_vector(user_w_write_8_data'range) := (others => '0');

begin
  xillybus_ins : xillybus
    port map (
      -- Ports related to /dev/xillybus_read_8
      -- FPGA to CPU signals:
      user_r_read_8_rden => user_r_read_8_rden,
      user_r_read_8_empty => user_r_read_8_empty,
      user_r_read_8_data => user_r_read_8_data,
      user_r_read_8_eof => user_r_read_8_eof,
      user_r_read_8_open => user_r_read_8_open,

      -- Ports related to /dev/xillybus_write_8
      -- CPU to FPGA signals:
      user_w_write_8_wren => user_w_write_8_wren,
      user_w_write_8_full => user_w_write_8_full,
      user_w_write_8_data => user_w_write_8_data,
      user_w_write_8_open => user_w_write_8_open,

      -- General signals
      PCIE_PERST_B_LS => PCIE_PERST_B_LS,
      PCIE_REFCLK_N => PCIE_REFCLK_N,
      PCIE_REFCLK_P => PCIE_REFCLK_P,
      PCIE_RX_N => PCIE_RX_N,
      PCIE_RX_P => PCIE_RX_P,
      GPIO_LED => GPIO_LED,
      PCIE_TX_N => PCIE_TX_N,
      PCIE_TX_P => PCIE_TX_P,
      bus_clk => bus_clk,
      quiesce => quiesce
   );

  process (bus_clk)
 
  begin
            wr_en <= user_w_write_8_wren;
            din <= user_w_write_8_data;
            user_r_read_8_eof <= user_r_read_8_empty and not(user_w_write_8_open);
           
            --From upper case to lower case
            if (user_w_write_8_data="01000001") then --A
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000010") then --B
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000011") then --C
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000100") then --D
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000101") then --E
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000110") then --F
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01000111") then --G
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001000") then --H
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001001") then --I
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001010") then --J
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001011") then --K
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001100") then --L
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001101") then --M
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001110") then --N
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01001111") then --O
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010000") then --P
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010001") then --Q
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010010") then --R
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010011") then --S
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010100") then --T
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010101") then --U
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010110") then --V
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01010111") then --W
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01011000") then --X
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01011001") then --Y
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01011010") then --Z
                din(5) <= not user_w_write_8_data(5);
               
            --From lower case to upper case
            elsif (user_w_write_8_data="01100001") then --a
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100010") then --b
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100011") then --c
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100100") then --d
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100101") then --e
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100110") then --f
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01100111") then --g
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101000") then --h
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101001") then --i
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101010") then --j
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101011") then --k
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101100") then --l
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101101") then --m
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101110") then --n
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01101111") then --o
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110000") then --p
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110001") then --q
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110010") then --r
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110011") then --s
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110100") then --t
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110101") then --u
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110110") then --v
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01110111") then --w
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01111000") then --x
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01111001") then --y
                din(5) <= not user_w_write_8_data(5);
            elsif (user_w_write_8_data="01111010") then --z
                din(5) <= not user_w_write_8_data(5);
            end if;
            --user_r_read_8_eof <= user_r_read_8_empty and not(user_w_write_8_open);
  end process;

--  8-bit loopback

  fifo_8 : fifo_8x2048
    port map(
          clk        => bus_clk,
          srst       => reset_8,
          din        => din,
          wr_en      => wr_en,
          rd_en      => user_r_read_8_rden,
          dout       => user_r_read_8_data,
          full       => user_w_write_8_full,
          empty      => user_r_read_8_empty
      );

    reset_8 <= not (user_w_write_8_open or user_r_read_8_open);

    --user_r_read_8_eof <= '0';
    --user_r_read_8_eof <= user_r_read_8_empty and not(user_w_write_8_open);
   
end sample_arch;


Here the main class to execute the writing thread

Code: Select all

import java.io.*;

public class TestWrite {
   
   public static void main(String [] args) {
      String inputPath = args[0]; //input file path
      WriteThread writer = new WriteThread(inputPath);
      new Thread(writer).start();                  
   }
}


Here the code of writing thread

Code: Select all

import java.io.*;

public class WriteThread implements Runnable {

   File inputFile=null;
      
   public WriteThread(String inputPath) {
      //Initialize Put
      this.inputFile = new File(inputPath);
   }

   @Override
   public void run() {
      
      int mb = 1048576;
      
      try {
         File writeDeviceFile = new File("/dev/xillybus_write_8"); //write device file
         InputStream inputFileStream = new FileInputStream(inputFile); //input stream for input file
         OutputStream writeDeviceFileStream = new FileOutputStream(writeDeviceFile, true); //output stream for write device file      
         byte[] bufPut = new byte[mb]; //buffer for copying from input file to write device file
         int lenPut = 0;
         int counter = 0; //debug

         //Write on xillybus_write_8
         while ((lenPut = inputFileStream.read(bufPut)) != -1 ) {//reads up bufPut.length bytes of data from inputFileStream (input file) into bufPut    
            writeDeviceFileStream.write(bufPut, 0, lenPut); //write from bufPut (input device file) to write device file
            writeDeviceFileStream.flush();
            counter++; //debug
         }
         inputFileStream.close();
         writeDeviceFileStream.close();
         System.out.println("Write counter: " + counter); //debug
      }
      catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in this directory.");
            System.exit(0);
      }
      catch (IOException e) {
            System.out.println(e.getMessage());
      }
   }
}


ere the main class to execute the reading thread

Code: Select all

import java.io.*;

public class TestRead {
   
   public static void main(String [] args) {
      String outputPath = "./output.txt"; //output file path
      ReadThread reader = new ReadThread(outputPath);
      new Thread(reader).start();                  
   }
}



Here the code of reading thread

Code: Select all

import java.io.*;

public class ReadThread implements Runnable{
   
   File outputFile=null; //output file   

   public ReadThread(String outputPath){
      this.outputFile = new File(outputPath);
   }

   @Override
   public void run() {
      
      int mb = 1048576;
            
      try {
         File readDeviceFile = new File("/dev/xillybus_read_8"); //read device file
         InputStream readDeviceFileStream = new FileInputStream(readDeviceFile); //input stream for read device file
         OutputStream outputFileStream = new FileOutputStream(outputFile, true); //output stream for output file
         byte[] bufGet = new byte[mb]; //buffer for copying from read device file to output file
         int lenGet = 0;
                  
         int counter = 0; //debug
         while((lenGet = readDeviceFileStream.read(bufGet)) != -1) {//reads up bufGet.length bytes of data from readDeviceFileStream (read device file) into bufGet
            outputFileStream.write(bufGet, 0, lenGet); //write from bufGet (read device file) to output file
            outputFileStream.flush();
            counter++; //debug
         }         

         readDeviceFileStream.close();
         outputFileStream.close();
         System.out.println("Read counter: " + counter); //debug      
      }
      catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in this directory.");
            System.exit(0);
      }
      catch (IOException e) {
            System.out.println(e.getMessage());
      }
   }
}


[support]

Hi,

Good to hear that it works now.

As for the different number of read and write loops, the short answer is that there is no reason to expect them to be equal. Or to have any particular number.

The thing is that both read() and write(), at the system call level, may act on less bytes than required to. I'm not familiar with how Java wraps these system calls, but it seems like it guarantees that all data that was given for write will be written (sooner or later, or upon a flush or close). It also seems like it takes the liberty to read less data than required -- which is why the caller of inputFileStream.read() gets the number of bytes actually read.

So if you got 12 loops of reading, that sounds sensible to me: During some of these read() calls, there was less data than required ready for reading, so you got less. This is normal behavior. It's in fact normal to get different amounts of data on each read() (and possibly a different number of loops) each time you try.

Bottom line -- can't see any problem.

Regards,
Eli

[openmpi]

Ok I understand now.

In order to take advantage from the architecture of my Xilinx Artix 7 ac701 I would like to set the buffer size of my Java apps as the amount of totale available memory on my FPGA. E.g.; suppose that the total memory available on my FPGA is 1 MB, I would like to set the buffer size of my Java app 1 MB too in order to use all the memory available on my FPGA. My question is, in which memory data will be moved once data goes from write device file to FPGA? Block ram, Distributed RAM, Shift register or somewhere else?

Thanks.

[support]

Hi,

There is no chance that the memory in the FPGA of any kind amounts to 1 MBytes. The data you've seen stored somewhere goes to DMA buffers, allocated in the computer's RAM. This size of the DMA buffers is configured on the IP core, which resides in the FPGA, but the memory isn't there.

The only good reason for using large DMA buffers is real-time considerations -- to make up for temporary situations during which the computer programs doesn't get CPU, letting the data flow go on from these DMA buffers. In a coprocessing application like yours, there is no practical reason to pay much attention to buffer sizes. I don't think you'll get any significant difference if you play with your data chunk size ("mb") between 64 kB and any larger figure.

Regards,
Eli