Pentaho Data-Integration

Asked

Viewed 989 times

0

I configured the PDI to access an FTP server and download a file in csv, so that’s okay, the problem is that inside the folder on the FTP server there will always be more file, example:

REC_PEND_FECH_COM20180219130059.csv
REC_PEND_FECH_COM20180219132200.csv
REC_PEND_FECH_COM20180219134000.csv
.
.
.

That is to always copy the most current file, how to do this?
Here I used shell script on Pentaho to do FTP and copy the files (Mget in folder), but I do not know how to specify to always capture the latest file within the directory, the file name is dynamic (REC_PEND_FECH_COMaaaammddhhmmss.csv)

Can you help me?

pentaho-kettle

2 answers

0


Here’s a topic that helped me solve the problem: Link

  • Hey there, Edvaldo! Please provide the solution in your reply, as this link you mentioned may be unavailable some day and we may be without the answer.

  • use a Select values step to convert the date metadata into the string format you need using the default "yyyymmdd". Then you can use this variable in your step Get a file with FTP as a ${YOUR_DATE wildcard}

  • you can edit your reply and add this information to it.

0

Through JOB STEP there are not many filters to be used, but it is possible to list the files of an FTP using step Get File Names in conjunction with VFS (Virtual File Systems), this way you can perform date modification filters and sorts, or even extract the timestamp from the file name to perform the filter.

EDIT: An example of use would be: ftp://master:[email protected]/local/ inserir a descrição da imagem aqui

Note also that some FTP uses the Passive mode setting, for these Ftps you will need to add the Parameter:

vfs.ftp.Passivemode = true

In ktr where VFS will be used.

  • How will GET File Names be linked with VFS ? It is possible to use Filezilla ?

  • Using Filezilla through PDI I don’t know how it would be done, I updated the answer with an example of VFS in Get File Names

  • It was to open an example, listing ? files are giving error.

  • What was the command you used in step Get File Names to access FTP ?

  • I didn’t get to use Get File Names because I don’t know how to implement it.

Browser other questions tagged pentaho-kettle

You are not signed in. Login or sign up in order to post.