Libraries: JSON Parser - page 2

 
Thanks for the lib.
 
I like
 

Sorry from everyone for not replying I wasn't really involved with MT5 in the last few years. Anyway here's a working link for the library: https://gist.github.com/erdeszt/527c4d6994b0834d458b

If you have questions/requests post them as comments on github I'm more than happy to answer them but I don't usually log in to this forum.

 
Dear all.

I am facing a strange behavior. I often read json with this lib but there is a problem if json contains an array full of objects. If the array grows above 10(9) items,

libary stopped working. [index] is too smal, an _array also ......


value = jo.getArray("data").getObject(i).getObject("refQuotation").getObject("quote").getDouble("value") ;

works until i becomes 10 of getObject.


Any ideas?



Thanks!
 
derdigge:

libary stopped working. [index] is too smal, an _array also ......

Please provide exact error message.
 
femto:

Hello,

 

I've found some bugs in this library.

Some help will be nice.

Unfortunatelly I sent an email to ydrol some days ago but I didn't get any reply about it.

 

JSON string was:

      string s="{ \"firstName\": \"John\", \"lastName\": \"Smith\", \"age\": 25, \"size\": 1.78, \"address\": { \"streetAddress\": \"21 2nd Street\", \"city\": \"New York\", \"state\": \"NY\", \"postalCode\": \"10021\" }, \"phoneNumber\": [ { \"type\": \"home\", \"number\": \"212 555-1234\" }, { \"type\": \"fax\", \"number\": \"646 555-4567\" } ], \"gender\":{ \"type\":\"male\" }  }";

 

First: 

isNumber() function doesn't return true when a number is in JSON string.

            JSONValue*jsv;

            jsv=jo.getValue("age");

            unittest.assertFalse(jsv.isNull()); // there is a bug here

            unittest.assertTrue(jsv.isNumber()); // there is a bug here

 

 

Second:

Safe access to an integer number in a JSON string doesn't  work as expected.

 

            int age = 0;

            if (jo.getInt("age",age) ) { // there is a bug here (return false... should return true)

               unittest.assertEquals(age, 25);

            } else {

               unittest.assertTrue(false, "Can't access age as int"); // there is a bug here       

            }

 

 

 

This library (and also hashmap implementation) is really interesting I think we should put it on GitHub in order to improve it.

I wrote some unittest code using https://github.com/femtotrader/mt4-unittest 

 

See as attachment a unittest code to show you problems.

 

I hope someone here can help. If I can't fix that I will only use string data in JSON. 

 

Kind regards 


The safe access to numbers doesn't work because the type isn't set in JSONNumber class. The fix is to add the following line in both JSONNumber constructors in json.mqh:

setType(JSON_NUMBER);

-Jozef

 
Jozef Bartek:


The safe access to numbers doesn't work because the type isn't set in JSONNumber class. The fix is to add the following line in both JSONNumber constructors in json.mqh:

Following a mention of the JSON parser in https://www.mql5.com/en/forum/227793...

I think there are at least three problems with the JSON parser as posted on this site:

  • Doesn't like empty strings
  • Doesn't like empty arrays
  • Doesn't do setType(JSON_NUMBER) on numbers, causing the problem identified by Jozef.

Attached version should sort out these three issues. There may be other, similarly minor things not yet covered.

Files:
json.mqh  32 kb
 

Hello,

I found some other bugs:

** When a string begins with an escape, e.g. "\n", or when two escapes succeed, e.g. "xxx\r\nyyy", the parser delivers nonsense as the parsed string, i.d. the rest of the JSON code.
   The reason for this is because the MQL4 function StringSubstr(str, i, n) that is used by the parser is a bit inconsistent: if n is zero, the whole rest of
   the string beginning from position i is delivered.

** Parser fails when parsing negative integers (function JSONParser::parseValue() )

** When serializing a string ( function JSONString::toString() ), the control codes (e.g. \n ) are not escaped


Missing features:

** Parse unicode escapes in strings (e.g. \u1234 )

** In the file hash.mqh the hash is now calculated from all 16 bits of unicode characters instead of only from 8 bits (ASCII-Strings).


Minor changes:

** A probably better hash algorithm is used (FNV-1a) at the cost of a little bit more computation time

** Make constructor of JSONValue protected in order to prevent that objects of this base class are created (e.g. the member variable _type of JSONValue

is not initialized in the constructor of JSONValue !).

**Although JSON has only the data type NUMBER, I'd like to store in a JSONNumber object the information whether the the JSON code from that

the JSONNumber object has been created contains an integer with or without a decimal point and a following zero (e.g. 0 vs. 0.0). For this, I added

an is_double - flag to JSONNumber. Without the flag one could not distinguish between 0 and 0.0 (but between other integers with or without decimal point).


I have corrected the bugs and added unicode escape parsing and changed the file hash.mqh in order to take 16 bit unicode character codes into account.

-- winloosewin

Files:
hash.mqh  23 kb
json.mqh  38 kb
 
The UnitTest library might help with development. https://www.mql5.com/en/code/11111 I use it sometimes when writing complex things like parsers.
Simple UnitTest include library for new MQL4
Simple UnitTest include library for new MQL4
  • www.mql5.com
This is a simple (cheap) UnitTest include library for MQL4. See a sample attacched: TestExpert.mq4 By UnitTest#printSummary(), the result is output as follows:
 

Update:

I added the following features to json.mqh:

*** In the JSONParser class I added counters for the line and column number of the current parse position.
In case of a parse error one can get these positions with JSONParser::get_line() and JSONParse::get_column().
This is the position where the parse error occured.

*** When parsing had no errors, the user will traverse the data structure delivered by JSONParse::parse().
There may be errors that came from a syntactically valid, but semantically invalid JSON file, e.g. if a positive number is expected but
a negative is detected - or if an integer is expected but a double-non-integer number is found.
In this case one wants also information about the position of the erroneous object in the JSON file. For this,
I have added line and column information in every JSONValue class that can be got by JSONValue::get_line() and JSONValue::get_column().

*** I added the functions isInt(), isIntType(), isLong(), isLongType(),isDouble() to the JSONValue class. The isLongType() delivers
only true if the JSONNumber was really created as an integer and not as a double that is convertible lossless to an integer.
In contrast, the isLong() function delivers also true if the JSONNumber has internally the is_double-flag set but is convertible lossless to
a long integer.

*** In JSONArray and JSONObject I added some convenience functions for type checking (and in JSONObject checking the existence of an object
with the specified key). The functions are e.g. JSONArray::isString(int index) and JSONObject::isString(string key).


--- Change to the original behavior: ---
In JSONValue and its subclasses have functions get_long() and get_int(). There are versions that return the long or int value
directly - even if the JSONValue is a JSONNumber with double type and if it's not convertible lossless to an integer.
If the JSONValue is no JSONNumber, the program will crash.
On the other hand there are get_long() and get_int()-versions that return a bool and have an additional parameter into that
the long or int result will be stored. In the original JSON.mqh version these functions will return true and write the result
if we have an JSONNumber object - regardless of whether a lossless conversion if possible.
But I think that normally the user wants to throw an error if an integer is expected and a non-integer number is found.
Therefor I changed the behaviour of these functions so that they will only return true and store the result if a lossless
conversion is possible.
The functions get_long_type() and get_int_type() that I added are even more strict: They only return true and store the
result if the JSONNumber object was created from an integer and not from a double type that is lossless convertible to an integer.

Files:
json.mqh  48 kb