How to scan log files for certain information?

Author

Message

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 16:03

Link

Hello,

I am stuck on a project I am trying to complete. I need to write a program that will scan log files (e.g log.txt) and search for specific strings of information. Mainly the information when someone connects to my server, such as

"L 05/29/2004 - 10:59:42: "freedomfighter<10><4566609><>" connected, address "555.555.555.555:26666"

After these strings of information are found I need them to be written to a new file. If anyone can help me here on how this might be achievable it would be greatly appreciated. Thanks!

~Falling Down~

Back to top

Profile PM Email Website

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 16:32

Link

our plugin has logging, file I/O, and a instr() function that should help you.

you can view the docs and download the plugin by clicking the logo below.

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 16:50

Link

Sorry but that all seems a bit advanced for me. I got the plugins installed but using them is a different story. Could you assist me more?

I am needing to capture the strings in a log file that only shows when someone connects not the rest. Below is a example of the log file.

L 05/26/2004 - 04:42:18: "torrente<17><4632279><>" connected, address "555.555.555.555:27005"
L 05/26/2004 - 04:42:34: "Ofay<13><0><13>" killed "Cutter<15><0><15>" with "9mmhandgun"
L 05/26/2004 - 04:43:10: "Cutter<15><0><15>" killed "Ofay<13><0><13>" with "crowbar"
L 05/26/2004 - 04:43:18: "Ofay<13><0><13>" killed "Chumly<10><0><10>" with "ARgrenade"
L 05/26/2004 - 04:43:42: "Ofay<13><0><13>" killed "Chumly<10><0><10>" with "9mmAR"
L 05/26/2004 - 04:44:40: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "satchel"
L 05/26/2004 - 04:45:48: "Bocefus<12><0><12>" killed "Ofay<13><0><13>" with "9mmAR"
L 05/26/2004 - 04:45:58: "Ofay<13><0><13>" killed "Cutter<15><0><15>" with "9mmhandgun"
L 05/26/2004 - 04:46:31: "Ofay<13><0><13>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:47:47: "Bocefus<12><0><12>" killed "Ofay<13><0><13>" with "9mmhandgun"
L 05/26/2004 - 04:47:48: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "tripmine"
L 05/26/2004 - 04:48:28: "torrente<17><4632279><17>" entered the game
L 05/26/2004 - 04:48:42: "Ofay<13><0><13>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:49:32: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 04:50:09: "Ofay<13><0><13>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:50:28: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:50:47: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:51:00: "Bocefus<12><0><12>" killed "Chumly<10><0><10>" with "9mmAR"
L 05/26/2004 - 04:51:14: "Ofay<13><0><13>" killed "Bocefus<12><0><12>" with "crowbar"
L 05/26/2004 - 04:51:15: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "9mmAR"
L 05/26/2004 - 04:51:25: "torrente<17><4632279><17>" committed suicide with "satchel"
L 05/26/2004 - 04:51:43: "KmartWarrior<11><0><11>" killed "Chumly<10><0><10>" with "shotgun"
L 05/26/2004 - 04:51:43: "KmartWarrior<11><0><11>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:51:52: "Bocefus<12><0><12>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:52:06: "Ofay<13><0><13>" killed "Cutter<15><0><15>" with "9mmhandgun"
L 05/26/2004 - 04:52:26: "Chumly<10><0><10>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:54:00: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:54:04: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 04:54:16: "KmartWarrior<11><0><11>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:54:58: "Ofay<13><0><13>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:55:36: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:55:58: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:56:07: "Ofay<13><0><13>" killed "Chumly<10><0><10>" with "9mmhandgun"
L 05/26/2004 - 04:56:23: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 04:57:33: "Chumly<10><0><10>" killed "Cutter<15><0><15>" with "9mmhandgun"
L 05/26/2004 - 04:58:21: "Chumly<10><0><10>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:58:31: "torrente<17><4632279><17>" killed "HoeCake<14><0><14>" with "9mmAR"
L 05/26/2004 - 04:58:46: "KmartWarrior<11><0><11>" killed "Chumly<10><0><10>" with "9mmAR"
L 05/26/2004 - 04:59:16: "HoeCake<14><0><14>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 04:59:30: "torrente<17><4632279><17>" killed "HoeCake<14><0><14>" with "9mmAR"
L 05/26/2004 - 05:00:14: "Chumly<10><0><10>" killed "torrente<17><4632279><17>" with "9mmAR"
L 05/26/2004 - 05:00:17: "Cutter<15><0><15>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:00:52: "Chumly<10><0><10>" killed "torrente<17><4632279><17>" with "shotgun"
L 05/26/2004 - 05:01:13: "Chumly<10><0><10>" killed "torrente<17><4632279><17>" with "9mmAR"
L 05/26/2004 - 05:01:41: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "shotgun"
L 05/26/2004 - 05:02:34: "[PNP]Scooby-Dum<18><4261213><>" connected, address "218.35.53.13:27005"
L 05/26/2004 - 05:02:39: "Chumly<10><0><10>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:03:46: "Cutter<15><0><15>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:04:26: "Cutter<15><0><15>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:04:52: "torrente<17><4632279><17>" killed "Cutter<15><0><15>" with "crossbow"
L 05/26/2004 - 05:05:22: "torrente<17><4632279><17>" killed "KmartWarrior<11><0><11>" with "crossbow"
L 05/26/2004 - 05:05:50: "torrente<17><4632279><17>" killed "HoeCake<14><0><14>" with "crossbow"
L 05/26/2004 - 05:06:10: "KmartWarrior<11><0><11>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:06:26: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 05:06:30: "Chumly<10><0><10>" killed "KmartWarrior<11><0><11>" with "shotgun"
L 05/26/2004 - 05:06:38: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 05:07:03: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"
L 05/26/2004 - 05:07:17: "KmartWarrior<11><0><11>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:07:20: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 05:07:41: "KmartWarrior<11><0><11>" killed "torrente<17><4632279><17>" with "shotgun"
L 05/26/2004 - 05:07:50: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "9mmhandgun"
L 05/26/2004 - 05:07:52: "HoeCake<14><0><14>" killed "KmartWarrior<11><0><11>" with "9mmAR"
L 05/26/2004 - 05:08:15: "torrente<17><4632279><17>" killed "HoeCake<14><0><14>" with "tau_cannon"
L 05/26/2004 - 05:08:50: "Ofay<13><0><13>" killed "torrente<17><4632279><17>" with "9mmhandgun"
L 05/26/2004 - 05:09:06: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "9mmAR"
L 05/26/2004 - 05:09:49: "Ofay<13><0><13>" killed "Chumly<10><0><10>" with "9mmhandgun"
L 05/26/2004 - 05:09:55: "KmartWarrior<11><0><11>" killed "Chumly<10><0><10>" with "9mmAR"
L 05/26/2004 - 05:10:19: "torrente<17><4632279><17>" killed "Ofay<13><0><13>" with "crossbow"
L 05/26/2004 - 05:10:29: "torrente<17><4632279><17>" killed "Cutter<15><0><15>" with "crossbow"
L 05/26/2004 - 05:10:47: "Ofay<13><0><13>" killed "KmartWarrior<11><0><11>" with "crowbar"
L 05/26/2004 - 05:12:22: "Ofay<13><0><13>" committed suicide with "worldspawn" (world)
L 05/26/2004 - 05:12:35: "Chumly<10><0><10>" killed "Cutter<15><0><15>" with "9mmAR"
L 05/26/2004 - 05:12:45: "Cutter<15><0><15>" killed "Ofay<13><0><13>" with "shotgun"
L 05/26/2004 - 05:12:46: "torrente<17><4632279><17>" killed "Chumly<10><0><10>" with "crossbow"

~Falling Down~

Back to top

Profile PM Email Website

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 17:25

Link

ok, I will work up some code that uses our plugin - back in a few...

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 18:04

Link

here ya go...

+ Code Snippet

`read the log file into a string
s as string
s=fil_getfiletostring("log.txt")

`split the string on crlf and return upper-bound
iUBound As Integer
iUBound = str_Split(s, chr$(13)+chr$(10), -1)

`create a string array based on upper-bound
Dim MyArray(iUBound) As String

`get each array element
For n = 0 To iUBound
MyArray(n) = str_GetSplit(n)

`test for unique item in string: "connected"
if str_instr(MyArray(n), "connected") <> 0
   fil_appendlog "connected_log.txt",MyArray(n)
   print "Added to log..."
   print MyArray(n)
endif

Next n

Wait Key

name your log file "log.txt"
the code above will find the lines that contain "connected" and write the whole line to an appended file named "connected_log.txt"

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 18:24

Link

I loaded your code and gave it a shot.

Its replying " Cannot preform 'string' cast on type '@fil_getfiletostring at line 8"

??

Maybe I didnt install the plugin correctly?

~Falling Down~

Back to top

Profile PM Email Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 18:32

Link

I must have extracted the plugins wrong. It seems to be working now.. I just have to figure out where it spit the scanned log file out at

brb

~Falling Down~

Back to top

Profile PM Email Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 18:35

Link

I can't seem to find where it spit the file out at. I dunno if it created a file at all?

~Falling Down~

Back to top

Profile PM Email Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 18:53

Link

It looks like its just "running" then exiting, as tho theres no file manipulation occuring. Hmm..

Or else I'm doing somthing really wrong..

~Falling Down~

Back to top

Profile PM Email Website

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 19:13 Edited at: 30th May 2004 19:16

Link

the file log.txt and connected_log.txt should both be in the directory with your exe, as well as the dbp_netlib.dll. also have you installed the .NET framework 1.1?

if you run the above code and then it just quits, something aint right because the last line of code is wait key

test with the following:

do
set cursor 0,0
print dt_GetNow()
loop

you should see a ticking clock. if not something's up.

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 19:20

Link

everything needed is in the program dir, I have framework 1.1, the test ran perfectally fine

~Falling Down~

Back to top

Profile PM Email Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 19:21

Link

For some reason, the commands your plugin's add don't show up as normal blue DBPro commands. I'm not 100% sure if they should but I believe there was reference to them turning blue like DBPro commands once there placed in the respective folders

~Falling Down~

Back to top

Profile PM Email Website

the_winch

21

Years of Service

User Offline

Joined: 1st Feb 2003

Location: Oxford, UK

Posted: 30th May 2004 19:22 Edited at: 30th May 2004 19:24

Link

Try this, uses native dbpro commands.

infile as string  : `input file name
outfile as string : `output file name

infile = "in.txt"
outfile = "out.txt"

s as string

open to read 1,infile
  if file exist(outfile) then delete file outfile
  open to write 2,outfile
    repeat
    `read a line from input file
    read string 1,s
    `check if the string has "connected" in it
    if contains(s,"connected") = 1
      `write the string to the output file if it has "connected in"
      write string 2,s
    endif
    until file end(1)
  close file 2
close file 1

function contains(in as string,find as string)
  `returns 1 if find is in in
  local out as integer : local length as integer : local i as integer
  for i = 1 to len(in)-len(find)
    if mid(in,i,len(find)-1) = find
      `found find in in so return 1
      exitfunction 1
    endif
  next i
  `if we make it here find not found in in so return 0
endfunction 0
  
function mid(str as string, start as integer, length as integer)
  `proper mid command
  local out as string : local i as integer
  for i = start to start + length
    out = out + mid$(str,i)
  next i
endfunction out

+ Code Snippet

infile as string  : `input file name
outfile as string : `output file name

infile = "in.txt"
outfile = "out.txt"

s as string

open to read 1,infile
  if file exist(outfile) then delete file outfile
  open to write 2,outfile
    repeat
    `read a line from input file
    read string 1,s
    `check if the string has "connected" in it
    if contains(s,"connected") = 1
      `write the string to the output file if it has "connected in"
      write string 2,s
    endif
    until file end(1)
  close file 2
close file 1


function contains(in as string,find as string)
  `returns 1 if find is in in
  local out as integer : local length as integer : local i as integer
  for i = 1 to len(in)-len(find)
    if mid(in,i,len(find)-1) = find
      `found find in in so return 1
      exitfunction 1
    endif
  next i
  `if we make it here find not found in in so return 0
endfunction 0
  
function mid(str as string, start as integer, length as integer)
  `proper mid command
  local out as string : local i as integer
  for i = start to start + length
    out = out + mid$(str,i)
  next i
endfunction out

he told me that he loved me and then he kicked me in the chest

Back to top

Profile PM Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 19:29

Link

HEY NICE! Thanks to both of you CattleRustler & the_winch. I've seen you guys posting all over the forums helping folks out

I'm glad theres so much supports here from people who know about the language.

Your code worked excellent the_winch. The new plugins will also give me more to learn

Maybe I'll eventually learn how to scan for strings with your plugins CattleRustler.

Thanks guys!

~Falling Down~

Back to top

Profile PM Email Website

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 19:58

Link

cool. The dbp_netlib_kw.ini file should be placed in the Editor\Keywords directory (if using dbp stock editor)

the doc files has everything you need as far as setup and usage

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

David T

Retired Moderator

22

Years of Service

User Offline

Joined: 27th Aug 2002

Location: England

Posted: 30th May 2004 20:05

Link

Maybe you need to add a few syncs to your code, CR?

Plus, if a comand doesn't apear blue it doesn't mean it's wrong - just that yoiu haven't copied the .ini file over yet.

Two strings walk into a bar. I'll have a pint says the first$%ASLDJ09920D"$"$D. Excuse my friend says the second, he isn't null terminated.

Back to top

Profile PM Email Website

Null Freq

20

Years of Service

User Offline

Joined: 16th Dec 2003

Location:

Posted: 30th May 2004 20:14

Link

I followed the instructions for CR's plugins to the T tho?

~Falling Down~

Back to top

Profile PM Email Website

CattleRustler

Retired Moderator

21

Years of Service

User Offline

Joined: 8th Aug 2003

Location: case modding at overclock.net

Posted: 30th May 2004 22:16

Link

the code above worked on my machine perfectly
I don't know whats different on your side

* DBP_NETLIB_v1.2 - NOW WITH VARIABLE WATCHER! * Click Logo

Back to top

Profile PM

Sorry your browser is not supported!

Newcomers DBPro Corner / How to scan log files for certain information?