one style tip would be compute the filenames rather than hard coding them.
so this,
if (letter = 0)
readFile = OpenToRead("/media/dictionary/A-Words.txt")
elseif (letter = 1)
readFile = OpenToRead("/media/dictionary/B-Words.txt")
elseif (letter = 2)
readFile = OpenToRead("/media/dictionary/C-Words.txt")
elseif (letter = 3)
readFile = OpenToRead("/media/dictionary/D-Words.txt")
elseif (letter = 4)
readFile = OpenToRead("/media/dictionary/E-Words.txt")
elseif (letter = 5)
readFile = OpenToRead("/media/dictionary/F-Words.txt")
elseif (letter = 6)
readFile = OpenToRead("/media/dictionary/G-Words.txt")
elseif (letter = 7)
readFile = OpenToRead("/media/dictionary/H-Words.txt")
elseif (letter = 8)
readFile = OpenToRead("/media/dictionary/I-Words.txt")
elseif (letter = 9)
readFile = OpenToRead("/media/dictionary/J-Words.txt")
elseif (letter = 10)
readFile = OpenToRead("/media/dictionary/K-Words.txt")
elseif (letter = 11)
readFile = OpenToRead("/media/dictionary/L-Words.txt")
elseif (letter = 12)
readFile = OpenToRead("/media/dictionary/M-Words.txt")
elseif (letter = 13)
readFile = OpenToRead("/media/dictionary/N-Words.txt")
elseif (letter = 14)
readFile = OpenToRead("/media/dictionary/O-Words.txt")
elseif (letter = 15)
readFile = OpenToRead("/media/dictionary/P-Words.txt")
elseif (letter = 16)
readFile = OpenToRead("/media/dictionary/Q-Words.txt")
elseif (letter = 17)
readFile = OpenToRead("/media/dictionary/R-Words.txt")
elseif (letter = 18)
readFile = OpenToRead("/media/dictionary/S-Words.txt")
elseif (letter = 19)
readFile = OpenToRead("/media/dictionary/T-Words.txt")
elseif (letter = 20)
readFile = OpenToRead("/media/dictionary/U-Words.txt")
elseif (letter = 21)
readFile = OpenToRead("/media/dictionary/V-Words.txt")
elseif (letter = 22)
readFile = OpenToRead("/media/dictionary/W-Words.txt")
elseif (letter = 23)
readFile = OpenToRead("/media/dictionary/X-Words.txt")
elseif (letter = 24)
readFile = OpenToRead("/media/dictionary/Y-Words.txt")
elseif (letter = 25)
readFile = OpenToRead("/media/dictionary/Z-Words.txt")
endif
Becomes this,
; compute the file name
FileName$= chr$(asc("A")+Letter)
Filename$= "/media/dictionary/"+FileName$+"-Words.txt"
; open file handle
readFile = OpenToRead(Filename$)
I suspect the loading is slow due to how spooling strings generally works, which tends to be byte by byte. Terrible for big files. Moreover string comparisons within the loop can be detrimental also.
One simple idea would be store the number of lines at the head of the document. Then you can use a FOR/NEXT block to grab the lines, without polling the device for the size all the file and no 'end' tokens.
ie.
LinesInFile = val(ReadLine(readFile) ) -1
for Index=0 To LinesInFile
Dictionary[letter, index]= ReadLine(readFile)
next
example file
50000
AAAAAAAA
ABBA
ABBADABBA
etc etc
That still a lot of string / disc thrashing though..
probably best to load the file into a memblock using
CreateMemblockFromFile
Rather than store the dictionary as 'text' files, i'd be more include to pre-process the file so that I can do searches directly on the memblock memory and not have to load it as strings at all, which introduces a lot of memory thrashing..
one idea would be to run through the files, for each file compute a table that is a life of byte offsets without the data block, so using the the header you can index the offset to the string you want. Storing a simple hash with the offset might be handy too, which will avoid doing string comparisons . The header could saving separately or attached to the text data, the latter is more messing around, but should be faster to load (slightly).
