Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / How do I read a CSV data file

Author
Message
Rudders
12
Years of Service
User Offline
Joined: 20th Jan 2012
Location: Oxfordshire UK
Posted: 5th Jul 2012 09:55 Edited at: 5th Jul 2012 09:57
Hi all, Can someone please give me an example of how to read data from a CSV file...

I have attached an example CSV file with with 3 records each with 28 fields of data.

Thanks to anyone for some help.

Regards

Rudders

Attachments

Login to view attachments
Neuro Fuzzy
16
Years of Service
User Offline
Joined: 11th Jun 2007
Location:
Posted: 5th Jul 2012 10:43
You can read it using the string commands. Open the .csv using notepad (or equivalently rename the file to .txt and double click)
This is what's in the file you posted:

You can use the getStringToken command to get the elements.

I'm unsure of the fine details of the csv format (what can be quoted, what can't, why there are a bunch of commans right after another), but just mention if you need more help than this.

JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 5th Jul 2012 11:03
The bunches of commas are empty fields in the grid.

CSV is a "loose" format. It's quite common to use TAB characters or semicolons instead of commas, because many regions use comma as a decimal point. It's important not to make assumptions.

Double quotes MUST be used if the separator character is in the actual string for a particular field. In the example above, the first line contains a field which is enclosed in quote marks because it has two commas in it. Consequently, getStringToken() cannot be used on its own to parse this file.

-- Jim
The Daddy
15
Years of Service
User Offline
Joined: 13th Jan 2009
Location: Essex
Posted: 5th Jul 2012 12:14
Hi

Is the file created by another application?

If not I would ignore the csv format and create a UDF that has the required fields in:

Type Tmyrecordtype
Name as string
Address as string
Order num as integer
Etc...etc
End type

Then simply write or read a file using readinteger or readstring. Hey Presto I like pesto.

Constantly seeking!
JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 6th Jul 2012 01:04
Have a look at the program uploaded here. This constructs a simple database out of the CSV file provided by Rudders, and should handle most others.

The zip contains:

* A Windows executable
* An Android APK file
* A uMain.pas file, which can be placed into a copy template project
* A uMain.rtf file, which is heavily-commented source
* A NOTES.rtf file which is er... notes

This works fine on Windows and Android, and I expect it would on Mac and iOS.

Please unzip to a folder. It won't work if you click the EXE in the zip file without unpacking. But then, you knew that!

-- Jim

Attachments

Login to view attachments
Lost Dragon
13
Years of Service
User Offline
Joined: 22nd Aug 2010
Location:
Posted: 6th Jul 2012 09:15 Edited at: 6th Jul 2012 09:18
Here is an example I made that opens a csv, reads the data, prints out what it reads, closes the file, and then loops back to do it again.

My example assumes the # of records is known and will never change. It also assumes that "," will always be used ONLY as a delimiter.

If you have records that have "," in the company name (for example) then your code will be more complicated. If your number of records vary then your code will be more complicated.

This is a pretty basic example that will hopefully get you started.

Hope it helps.

(And I hope I didn't botch it too much)

Attachments

Login to view attachments
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 6th Jul 2012 12:05
If you must use this file format, then you could also wash the data first, which would be along the lines of:

1. read data
2. Convert commas within quotes to another unused character, usually an escape code rather than a keyboard character
3. Remove the quotes
4. Parse the data
5. Convert characters in step 2 back to commas.

JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 6th Jul 2012 13:38
Using the technique I posted above, it's only necessary to massage spaces - commas in quotes are handled automatically.

-- Jim
MarcoBruti
12
Years of Service
User Offline
Joined: 20th Nov 2011
Location: Caput Mundi
Posted: 6th Jul 2012 15:48
why you have to convert commas into unused chars? I never do it, if quotes are used for each field, it is enough to tokenize the string using quotes as delimiter, throwing away the intermediate tokens containing commas.
E.g.
"Marco","Bruti","1000,000"
Tokenize using " as delimiter:

Token0: empty
Token1: Marco
Token2: ,
Token3: Bruti
Token4: ,
Token3: 1000,000

Throw away token0, 2 and 4 (even), and that'all falks.
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 6th Jul 2012 16:28
Quote: "Using the technique I posted above, it's only necessary to massage spaces - commas in quotes are handled automatically."


nice, I'll check it out later.

Quote: "why you have to convert commas into unused chars? I never do it, if quotes are used for each field, it is enough to tokenize the string using quotes as delimiter, throwing away the intermediate tokens containing commas.
"


In your example, all data has quotes. In the original, only some data has quotes. So you need a method to handle conditionally embedded tokens.

MarcoBruti
12
Years of Service
User Offline
Joined: 20th Nov 2011
Location: Caput Mundi
Posted: 7th Jul 2012 14:22
JSON format requires that everything must be delimited, numbers and strings. JSON is not CSV but, even in CSV, in my humble opinion, it is not a good practice to have something double quoted and something not, strange things may occur.
Anyway you could extend my methodology, starting from the current index, searching twice " (the second search starting from the next char of the first ") and once for ,
Here are the case:
- If the position of , is before double quotes (or no double quotes exist), then the current field is not delimited.
- if the position of , is in the middle, then the comma is part of the field delimited by double quotes.
- if the position of , is after the double quotes, or comma does not exist, then the field is double quoted but it does not contain commas.

Easy, isn't it? I have not tried it, but I am 99% sure that it works.

Login to post a reply

Server time is: 2024-05-04 11:53:02
Your offset time is: 2024-05-04 11:53:02