Integer64 fields read as strings

[marcosvital]

Hi!

I'm working with QGIS 3.8.1 under Windows 10. I've created a "species richness grid" using the 'biological records tool' from FSC tools plugin, so I have a vector containing number of species inside grid squares.
When I used your plugin to call R and create a histogram based on the values from the vector layer, I've noticed that the values were wrong, and I've noticed the message "Integer64 fields read as strings".

Searching a little bit about it, Ive found this closed issue from someone with a similar problem: qgis/QGIS#30493

Is this a bug, something that could be corrected on a future release? Or am I doing something wrong here?
I've managed to work around this problem by converting the vector into a raster. It works, but then you would have to create several rasters if you want work with more then one layer field.

Let me know if you need any more info, ok?
Cheers
Marcos

[nyalldawson]

This is normal r behaviour, not anything specific to this plugin.

Have a read over the integer64 section on https://www.rdocumentation.org/packages/rgdal/versions/1.4-4/topics/readOGR for explanation.

We could change to warn.loss mode here, but my gut instinct is to stay with the default behaviour set upstream...

[marcosvital]

OK, thanks, I'll check the documentation.

Now, what would be the best way to work around it and get the layer to be read correctly? I've managed to get values right by converting to a raster, but I'm not sure if this is the best solution or if it would be always suitable...

[nyalldawson]

I would say you could cast the required field values to int when using them.

The alternative is if we expose this as an option for the input. But that makes me nervous, because (as the upstream r conversation discusses) this could lead to silent data loss...

[marcosvital]

So, I just tried changing the field values to int (using the field editor from the processing toolbox), but it didn't work right. I mean, changing the field to int seems to have worked, but noting changed when the values were read by the R script.

I double checked the layer properties, and the field is now int - it's the "Abundance" field on the image bellow:

When I call the R function and use str() on the field, it is identified as a factor:

Factor w/ 8 levels "1","11","13",..: 1 7 1 8 3 1 6 1 1 1 ...

I tried to force it to be numeric with as.integer() and again with as.numeric(), but it got the values from the factor codes, not the original numbers.

I feel that I might be missing something very simple here, but QGIS is still pretty new for me, sorry...

[nyalldawson]

I would suggest grabbing the actualy R script which is output when you run the algorithm, and then run this script in R alone. Get the script working as desired when it's within R, and only then work on getting the processing wrapper around your script working correctly.

[marcosvital]

I tried to read the dbf file from the layer (used read.dbf function from foreign package): data went fine into R, and the values were recognized as integers, so everything works from this point on.

[JanCaha]

Just one note. Reading the dbf file of the layer is not the same as reading the layer using "rgdal" package. Packages "foreign" and "rgdal" use completely different approaches to data reading processes.

[marcosvital]

You are right, @JanCaha , thanks!

OK, I tried different approaches to handle this, as for some unknown reason I couldn't change the field layer type using the 'refactor field' tool. May I was using it wrong, not sure...

But the problem was solved when I tried to use the field calculator do create a new field, using the values from the original field I wanted to use and setting the field type to integer. So problem solved for me, in a simple and easy way.

Thanks for the help, @nyalldawson and @JanCaha .

On a side note: is there official documentation for this plugin somewhere? I fund some useful info here: https://docs.qgis.org/testing/en/docs/training_manual/processing/r_intro.html
But it seems that some features described over there don't actually work (such as setting table as input).
I would be happy to contribute with the documentation if I can.

[nyalldawson]

OK, I tried different approaches to handle this, as for some unknown reason I couldn't change the field layer type using the 'refactor field' tool. May I was using it wrong, not sure...

It's likely dependant on what format you're storing the result in. There's a few formats which will silently upgrade int fields to long in some circumstances, and I suspect you're hitting these.

On a side note: is there official documentation for this plugin somewhere? I fund some useful info here: https://docs.qgis.org/testing/en/docs/training_manual/processing/r_intro.html

Actually those docs don't belong in the qgis manual at all (since it's now a plugin), and we should move them over to this repo and start on our own proper documentation.

[JanCaha]

Actually those docs don't belong in the qgis manual at all (since it's now a plugin), and we should move them over to this repo and start on our own proper documentation.

I there any preferable way how to do that? I have experience with Jekyll and Github pages, so if something like that would do the work then I could take a look at it and possibly do it.