Discussion:
Timestamp for input file
(too old to reply)
Sergey P
2017-10-01 13:35:59 UTC
Permalink
Hi, guys.

I want to change my input .csv file by timestamp in order to avoid to
repeated running by AL. I mean that AL will not run on file that have
timestamp, and conversely if file have not timestamp in filename - will be processed.

So my questions is:
1.Are that good idea? If so, this is possible to implement without deleting original input file and creating new as output file with same name but with timestamp in name?
( like :

task.log("Configuring File Name for File System Connector");
var srcPath="C:\Temp\"
var stDateStamp=system.formatDate((new Date()),"yyyyMMdd");

var inFile=srcPath + "fileSysInput_" + stDateStamp + ".csv ";
task.log("Input File Name: " + inFile);
thisConnector.setConnectorParam("filePath",inFile);

)?
2. If not, have anyone any proposal for this task? I know lazy and more simple way for this solution - just delete file from directory after AL was run on, but really I want elegant solution )


Thanks in advance for answers

Sergey
Eddie Hartman
2017-10-02 09:49:13 UTC
Permalink
Post by Sergey P
Hi, guys.
I want to change my input .csv file by timestamp in order to avoid to
repeated running by AL. I mean that AL will not run on file that have
timestamp, and conversely if file have not timestamp in filename - will be processed.
1.Are that good idea? If so, this is possible to implement without deleting original input file and creating new as output file with same name but with timestamp in name?
task.log("Configuring File Name for File System Connector");
var srcPath="C:\Temp\"
var stDateStamp=system.formatDate((new Date()),"yyyyMMdd");
var inFile=srcPath + "fileSysInput_" + stDateStamp + ".csv ";
task.log("Input File Name: " + inFile);
thisConnector.setConnectorParam("filePath",inFile);
)?
2. If not, have anyone any proposal for this task? I know lazy and more simple way for this solution - just delete file from directory after AL was run on, but really I want elegant solution )
Thanks in advance for answers
Sergey
You can use Java methods to do these sort of things, Sergey. Have a look at java.io.File, in particular the renameTo() method. Just be sure the File Connector is terminated (e.g. After Close) before you try this.

-Eddie
Sergey P
2017-10-02 14:34:09 UTC
Permalink
Hi Eddie,
and thank for hint. :)
Unfortunately , Java method 'renameTo( )' on 'java,io.File ' not work for me,
but I was use 'system.renameFile(oldFile,newFile)' that work fine.

Below I posted my code, in case someone else will be interested.


task.logmsg("Preparing to change input file name by timestamp");
var origPath = thisConnector.getConnectorParam("filePath");
var stDateStamp = system.formatDate((new Date()),"yyyyMMdd_HHmmSS");
var chngPath = origPath.replace(".csv","_" + stDateStamp + ".csv");
if (!system.renameFile(origPath,chngPath )){
task.logmsg("FAILED to rename the input file");
task.logmsg("The file name still ---> " + origPath );
}
else {
task.logmsg("SUCCESS to rename the input file");
task.logmsg("New file name is ---> " + chngPath );
}
Sergey P
2017-10-02 14:39:25 UTC
Permalink
Hi Eddie,
and thank for your hint. :)
Unfortunately , Java method 'renameTo( )' on 'java,io.File ' not work for me,
but I used 'system.renameFile(oldFile,newFile)' that work fine.

Below I posted my code, in case someone else will be interested.


task.logmsg("Preparing to change input file name by timestamp");
var origPath = thisConnector.getConnectorParam("filePath");
var stDateStamp = system.formatDate((new Date()),"yyyyMMdd_HHmmSS");
var chngPath = origPath.replace(".csv","_" + stDateStamp + ".csv");
if (!system.renameFile(origPath,chngPath )){
task.logmsg("FAILED to rename the input file");
task.logmsg("The file name still ---> " + origPath );
}
else {
task.logmsg("SUCCESS to rename the input file");
task.logmsg("New file name is ---> " + chngPath );
}
yn2000
2017-10-02 15:17:09 UTC
Permalink
"...I want elegant solution..." So, my solution is not elegant??? I am offended (just kidding) and here is the big picture of my solution.
I created input, output, archive, pending, and transaction folder. This folders are part of the process. When the file arrive it goes to input folder. Then I process the file. All errors go to the file in the pending folder. In my situation I need to create another file (with added data that is not part of the input file) to be process by other, so that I generate a file and send to output folder. Once completed, I copy the original input file from the input folder into archive folder, and yes, with a timestamp, and then delete the file in the input folder. All of my actions are being recorded (audited) in the transaction folder. And my timestamp is generated in the beginning of the AL, so that the whole process has the same timestamp.

So, I think my idea is more elegant than yours (by just adding a timestamp in the input file), because over time, the files are piling up in the folder and when you have too many files in the same folder, you will have difficulties to spot which files have not been processed.

:-) Rgds. YN.
Eddie Hartman
2017-10-03 08:43:03 UTC
Permalink
Post by yn2000
"...I want elegant solution..." So, my solution is not elegant??? I am offended (just kidding) and here is the big picture of my solution.
I created input, output, archive, pending, and transaction folder. This folders are part of the process. When the file arrive it goes to input folder. Then I process the file. All errors go to the file in the pending folder. In my situation I need to create another file (with added data that is not part of the input file) to be process by other, so that I generate a file and send to output folder. Once completed, I copy the original input file from the input folder into archive folder, and yes, with a timestamp, and then delete the file in the input folder. All of my actions are being recorded (audited) in the transaction folder. And my timestamp is generated in the beginning of the AL, so that the whole process has the same timestamp.
So, I think my idea is more elegant than yours (by just adding a timestamp in the input file), because over time, the files are piling up in the folder and when you have too many files in the same folder, you will have difficulties to spot which files have not been processed.
:-) Rgds. YN.
Sounds like a truly elegant solution, YN :D
rpuggal
2017-12-21 02:29:12 UTC
Permalink
Hi yn, your solution elegant enough that I would like to see a sample if you don’t mind sharing ! I am. Doing something similar for a solution I am developing but not quite as high tech as what you stated, I could use some help on processing pending transactions from an input file that failed for whatever reason( updates failed , bad data), the retry logic with a file based iteration approach is somewhat lacking...
yn2000
2018-02-07 21:07:56 UTC
Permalink
Sorry, I just read the post...
The main ingredient of the code is just copy and delete scripts as the following:

//To copy a file =====
fromFile = new java.io.File("c:\\temp\\from.txt");
toFile = new java.io.File("c:\\temp\\to.txt");
system.copyFile(fromFile, toFile, true);

//To delete a file =====
delFile = new java.io.File("c:\\temp\\tobedeleted.txt");
system.deleteFile(delFile);

So, when you design the AL, you incorporate the file systems as part of your solution, such as you manipulate the file system before the Iterator (of the normal AL) begins. I started small, but then it grows. That is why naming convention for the file is very important. FYI, I also include date stamp here and there in the file name for easier perusal. Now, to answer your question, I do not accommodate 'retry' logic. On error, I asked the human/operator to intervene the process, because an error caused by a bad data is very poisonous. You cannot (or should not) 'retry' any process caused by bad data, because your solution could be looping (retrying to send bad data again and again).

HTH. YN.
rpuggal
2018-02-21 19:36:20 UTC
Permalink
Thanks yn very helpful info indeed. I will give it a try and let you know how it goes.
Loading...