Handling conversion errors - File Encoding

By Sarveswararao Sombhatla, Yash Technologies  

 One of the major effects that need to be taken care when reading the file from Application Server  

File encoding format  

What is Encoding?  

Encoding means converting the message into different format or symbols.  

Example: When we create a notepad and save it, by default the Encoding format will be ANSI. When we reopen the file it reads the characters as per the encoding format we saved and shows us the same characters what we have actually written in that notepad.  

ANSI Encoding  

Below is the screenshot how notepad file saved in ANSI Encoding format with a line ‘ABC’ is interpreted in a Hex-Editor.  

Here we have the simple hexadecimal conversion. 41 is A, 42 is B, 43 is C.  

Unicode Encoding  

The same file when saved in Unicode Encoding format, then how this gets interpreted in Hex-Editor  

We can see some additional hexadecimal characters when a file is stored in Unicode Encoding format.  

FF FE is the signature of the file encoded in Unicode or UTF-16 Big Endean format  

UTF-8 Encoding  

The same file when saved in UTF-8 Encoding format, then it gets interpreted in Hex-Editor as below:  

EF BB BF is the signature of the file encoded in UTF-8 format  

This way with the first additional characters of the files we can identify what is the encoding format used in this file.

What happens when a file with UTF-8 format and ANSI format file is placed on the SAP Application server directly?  

1. If a file saved with encoding UTF-8 format is placed on the SAP Application server, the file is read in AL11 as shown in the below screen-shot.  

We can see the special characters as they are because, by default SAP uses UTF-8 encoding format.

When we read this in ABAP program using OPEN DATASET statement with the addition ENCODING DEFAULT we can read the file with the special characters without any code page conversion problem.  

2. If a file saved with encoding ANSI format is placed on the SAP Application server, the file is read in AL11 as shown in the below screen-shot.  

In this case we are not able to see the special characters as they are encoded in ANSI format. When we read this file in ABAP program using OPEN DATASET statement with the addition ENCODING DEFAULT the program goes for a runtime error as the encoding format used by SAP is different from the encoding format the file is saved. We need to use OPEN DATASET statement with the addition ENCODING NON-UNICODE to read this file so we won’t get any runtime error.

To avoid this kind of runtime errors when we read the file from application server in SAP, we need to have always the same encoding format file to be placed on the Application server.