Data import with FileImporter & BulkImporter

What this is about

ESL-Easy imports data from Excel, CSV and XML sources. Two modules are available for this: the FileImporter and the BulkImporter. The process is the same for both:

You place a file in a monitored folder. ESL-Easy processes it automatically, detects what has changed compared to the previous import, and updates all labels linked to that data.

---

Step 1, FileImporter or BulkImporter?

This decision is made before configuration, because it determines which INI sections you need.

Rule of thumb: Large master data volumes imported cyclically → BulkImporter. Import plus control commands → FileImporter.

---

Step 2, Configure the INI file

The operating parameters are stored in ESLEasy_FileImporter.ini or ESLEasy_BulkImporter.ini.

screenshot002.png

The base sections

[LOOP]
Interval=20                 ; Sekunden zwischen den Prüfläufen

[PATH]
FilePath=C:\ESLEasy\ESLEasy_FileImporter\Files
ProcessedPath=C:\ESLEasy\ESLEasy_FileImporter\Files\Ablage
ErrorPath=C:\ESLEasy\ESLEasy_FileImporter\Files\Fehler

[IDF]
Encoding=UTF8
IdfNeeded=TRUE

[LOG]
LogLevel=1

• [LOOP] Interval=, the interval in seconds at which the program checks for new files.
• [PATH], the three working folders. FilePath is the monitored import folder, ProcessedPath the destination for successfully processed files, ErrorPath for failed ones. These three folders are the core of the process.
• [IDF] Encoding=, encoding of the IDF file. IdfNeeded=TRUE requires that a matching IDF or the Master.idf is present.
• [LOG] LogLevel=, scope of logging. The import history is stored in a daily log file in the .\Log folder.

The module-dependent block

FileImporter only, connection to the server:

[ESLEASY_SERVER]
IP=127.0.0.1
Port=49890
User=IXOP
Pass=POXI
EncodingRead=UTF8
EncodingWrite=UTF8
Encryption=false

BulkImporter only, connection to the database:

[SQL]
Type=Firebird
DateTimeFormat=dd.MM.yyyy HH:mm:ss

[FIREBIRD]
ConnectionString=Server=127.0.0.1;Port=49125;Charset=UTF8;User ID=SYSDBA;Password=<PASSWORD>;Database=C:\ERP\ERP.FDB
passencrypted=TRUE

Alternatively, a ConnectionString for Microsoft SQL Server is available under [MSSQL].

---

Step 3, The folder process

This is the core of the system, and it is deliberately kept simple.

screenshot003.png

screenshot004.png

Time-controlled import

If the file name ends with @ plus a timestamp in the format yyyyMMddHHmmss, the file remains in the import folder until that time is reached.

MasterData@20231102170000.xlsx

→ Import only on 02.11.2023 from 17:00.

The masks used can be adjusted in the INI under [IMPORTTIMESTAMP] (defaults: TimestampMask=yyyyMMddHHmmss, TimestampSeparator=@).

---

Step 4, File structure and required fields

For ESL-Easy to use the data, each record needs at least one key field: the __DATAID. In addition, a description should be provided as __DESCRIPTION.

There are two approaches:

• Directly in the file: Simply name two columns __DATAID and __DESCRIPTION, they are processed automatically.
• Via the IDF file: The usual case, because the ERP export usually cannot be changed. You define the mapping in the IDF (e.g. “field Artikel serves as __DATAID”).

screenshot005.png

---

Step 5, The IDF file

The IDF is the real lever. You need it whenever the data file is not directly processable (e.g. the __DATAID column is not recognizable) or when data must be modified during import.

The starting problem

Our sample file does not contain clean values, but text with embedded numbers:

"Art-Nr. 10101","Becks Pils 24 x 0.33 l MW","Inhalt/Fl.: 0,33","EUR/l     1,89","zzgl. Pfand EUR:     3,42","EUR/Gebinde:     14,99"

ESL-Easy cannot work with this. A header is missing, the text portions must be removed, and the article number must be recognized as __DATAID. That is exactly what the IDF does.

screenshot006.png

The IDF section by section

[CSV], define separator, quotation marks, header

[CSV]
Separator=,
QuotationMark=TRUE
Header="ArtNr","Beschreibung","Inhalt","Grundpreis","Pfand","Preis"

With QuotationMark=TRUE, all fields (header and data) must start and end with the QuotationMarkChar. The characters are removed during import.

[DATA], control import behaviour

[DATA]
DeleteBeforeImport=FALSE
TrimAll=TRUE
UpdateLabels=TRUE

[ADDTAG], add the required fields

[ADDTAG]
__PROGRAM=PUTDATA
__DATAID=<ArtNr>
__DESCRIPTION=<Beschreibung>
__STARTTIME=%%TODAY%%

• __PROGRAM=PUTDATA tells the server that the payload data should go into the product database, this is the normal data import. Further commands: PUTDATALOCATION, PUTREFERENCE, UPDATEDATAID, ADDQUEUETASK. Note: __PROGRAM applies only to the FileImporter.
• __DATAID=<ArtNr>, the angle brackets refer to a column of the import data. Multiple columns and free text can be combined.
• __STARTTIME=%%TODAY%%, processing time. Without a value: immediately.

[REPLACEVALUE], remove the text portions

[REPLACEVALUE]
ArtNr=Art-Nr.|
Inhalt=Inhalt/Fl.:|
Grundpreis=EUR/l|
Pfand=zzgl. Pfand EUR:|
Preis=EUR/Gebinde:|

The syntax is search text|replacement. We leave the replacement empty, so Inhalt/Fl.: is replaced by nothing, and the pure value 0,33 remains.

Further IDF sections at a glance

You do not need these sections for the first import, but sooner or later you will:

---

Step 6, The first import

Place both files, Artikel.csv and Artikel.idf, in the FilePath folder. Then wait.

screenshot008.png

screenshot009.png

ESL-Easy has detected the change and updated all labels linked to this DataID.

---

Step 7, Automate the import

In practice, nobody copies files by hand. Both modules can fetch import files themselves at defined times.

HTTP download

[HTTP]
Active=TRUE
StartTimes=01:00,02:00,03:00,16:15
;StartMins=00,30
URL=https://www.domain.com/Data
FileName=Import.csv
DownloadType=DATA
Encoding=UTF8
AutoDetectEncoding=FALSE
Security=TRUE

screenshot010.png

Further transport methods

• [FTP] / [SFTP], download before import. SFTP authentication via user/password, via private key file (.pem) or via X509Certificate.
• [ZIP], If the import data is delivered as a ZIP, ESL-Easy unpacks it automatically. ProcessFiles=*.csv filters which files are made available; DeleteAfterUnzip=TRUE deletes the archive afterwards.

---

Is file import the right approach?

ESL-Easy offers several data connection methods. For orientation:

The methods are combinable: ESL-Easy always merges local and external data into one record.

---

Common pitfalls

The file does not disappear from the import folder.

Processing was aborted, usually because of a connection problem (incorrect ConnectionString for BulkImporter, server unreachable for FileImporter). The file will be processed again on the next run. Check the log.

The import runs, but the labels do not change.

Check [DATA] UpdateLabels=TRUE in the IDF.

Field names are not recognized.

Field names must not begin with two underscores and must not contain any of the characters < > " * \.

Excel import fails.

Check the program version: Only from 1.2.0.0 does the importer work without a locally installed Excel.

---

Further reading

• Complete IDF reference, all sections and parameters: original PDF ESLEasy_FileImporter und ESLEasy_BulkImporter 2.3_EN, pp. 10–22.
• XML import, including CDATA and TEXT: same document, pp. 22–24.
• Next guide: Your first template in 15 minutes