compactframework >> Working with sorted fixed length text files ?

by Thomas » Tue, 12 Jul 2005 15:51:29 GMT

Hello !

I must develop an application for a WindowsCE.NET PDA with the .NET
Compactframework.

Due to compatiblity reasons with the backoffice application, the PDA
will have to receive and send back sorted fixed length text files (1
record per line, sorted in use order). I must precise that, at least
for one of the files, the number of records can be 10 000 max.

My questions regarding this situation are :
- would it be interesting to transform these files in XML files and
then to use Dataset / Datatable to manipulate it ?
- would it be interesting to transform these files in XML files and
then to use directly XML ? In this case, what about the methods to
search data and the performances ?
- or, last solution, since the files are sorted, could I use directly
text files ? At the moment, I think it would be the best solution but I
don't know the best way to search in such a file. If someone could give
me some clues or samples, it would be great.

I'm waiting for your advices, guys !

Regards,

Thomas.


compactframework >> Working with sorted fixed length text files ?

by Peter Foot [MVP] » Tue, 12 Jul 2005 16:08:23 GMT


If the files you are dealing with are large, and already in the right order
for you to process it wouldn't be that beneficial to read the whole file
into memory into XML or DataSet for example. However it can be done, take a
look at the OpenNETCF.Data.Text.TextDataAdapter (www.opennetcf.org/sdf/) for
example
Since your data is in lines you can create a StreamReader with your
filename, then call ReadLine to pull out each row. Then you can split this
using your delimiting character e.g. comma, tab etc to get your array of
fields. You may then parse these to specific types. Once you have then done
whatever processing you require on that row you could concatenate your
values back into a string for each row and write them to file using a
StreamWriter, then repeat the process for each row.
As I said previously I'd avoid reading in the whole file if possible,
however if you go through the source code for the TextDataAdapter you should
find parts of it are useful. Source code is in the vault
http://vault.netcf.tv/VaultService/VaultWeb/login.aspx (username: guest,
password guest)

Peter

--
Peter Foot
Windows Embedded MVP
http://www.inthehand.com | http://www.peterfoot.net |
http://www.opennetcf.org

compactframework >> Working with sorted fixed length text files ?

by Thomas » Tue, 12 Jul 2005 16:20:48 GMT

Thank you Peter for your quick answer.
If I decide to use text files, what is the best/speedest method to go
to a specific line/record ? Do I have to create a StreamReader and then
to read each line to find the first good one ?

Thank you.

compactframework >> Working with sorted fixed length text files ?

by » Tue, 12 Jul 2005 19:27:25 GMT

If it's already sorted, and fixed length it's easy. Use a StreamReader and
a binary search to find the record you're after. With 10k lines it will be
a maximum of 13 reads to find any desired record.

-Chris

compactframework >> Working with sorted fixed length text files ?

by Thomas » Tue, 12 Jul 2005 21:52:20 GMT

Thanks again. Could you give some links or some explanations about the
way to implement a binary search on a text file ?
Please apology but I've never worked with this type of files...

Regards,

Thomas.

compactframework >> Working with sorted fixed length text files ?

by Paul G. Tobey [eMVP] » Tue, 12 Jul 2005 23:49:00 GMT

Just like a binary search on anything else. Figure out how many records
there are by looking at the file size and the length of the record. You
might wrap the retrieval of record #x so that you don't have to deal with
the record size calculation all over the place. Set your pivot variable to
1/2 way through the total number of records and read that record. If the
value you're searching for is less than the current record, move the pivot
to 1/2 way between the first record and the current pivot and set the last
record to the current pivot -1. If the value is greater than the current
pivot, set the first record to the current pivot +1 and set the new pivot to
1/2 way between that and the end record. Repeat until the end record number
is less than the first record number or until you've found the record you're
searching for.

Paul T.

compactframework >> Working with sorted fixed length text files ?

by Thomas » Wed, 13 Jul 2005 20:48:25 GMT

OK, Thank you everybody for your precise answers. I will try to use
binary search maybe combined with datatable when I will have to read a
set of records.

Regards,

Thomas.

Similar Threads

1. How to get the fixed length of a fixed length string

Private Structure TNavRec

<VBFixedString(100),System.Runtime.InteropServices.MarshalAs(System.Runtime.
InteropServices.UnmanagedType.ByValTStr,SizeConst:=100)> Public strItemName
As String

<VBFixedString(4),System.Runtime.InteropServices.MarshalAs(System.Runtime.In
teropServices.UnmanagedType.ByValTStr,SizeConst:=4)> Public strItemID As
String

<VBFixedString(2),System.Runtime.InteropServices.MarshalAs(System.Runtime.In
teropServices.UnmanagedType.ByValTStr,SizeConst:=2)> Public EndOfLine As
String

End Structure

--snip--
Dim ludtNavRec As TNavRec
'accept more than a record will hold
cboName.Text = VB.Left(Value, Len(ludtNavRec.strItemName))

------
In my program
 Len(ludtNavRec.strItemName)
 returns 0 (probably because I haven't  stored anything into it yet).
Probably returning the length of the variable string not  the capacity.

I could, of course, store the max length (100) in a const and use that but
 I wonder if there is a better way?

Thanks


2. ReportViewer with fixed length text file - Asp.Net

3. Create fixed length text file

I have to create a fixed length text file and was hoping 
someone could point me to an example or resources that 
could help explain how it is done in C#.

Thanks.

4. importing a fixed field length formatted text file into a dataset - VB.Net

5. Reading Fixed Length ASCII/Text File

I have the following code to read the following file (very simple). 
I'm trying to extract only the date, but with this code I'm extracting
the entire record.  What am I doing wrong?  What am I missing?

Ken

c:\temp\namedate.txt = Ken Hall08/27/56 (One record)

Module Module1
  
    Public Structure Record1
        <VBFixedString(8)> Public name As String
        <VBFixedString(8)> Public date1 As String
    End Structure


    Sub Main()
        Try
            Dim line As String
            Dim Record2 As New Record1

            ' Create an instance of StreamReader to read from a file.
            Dim sr As System.IO.StreamReader = New
System.IO.StreamReader("c:\temp\namedate.txt")
            ' Read and display the lines from the file until the end 
            ' of the file is reached.
            Do
                line = sr.ReadLine()
                Record2.date1 = line
                Console.WriteLine(Record2.date1)
            Loop Until line Is Nothing
            sr.Close()
        Catch E As Exception
            ' Let the user know what went wrong.
            Console.WriteLine("The file could not be read:")
            Console.WriteLine(E.Message)
        End Try


    End Sub

End Module

6. how to split a string using ,fixed character length, variable text delimmiter - CSharp/C#

7. Transform fixed length text to xml

can i transform fixed length text to xml using XSL
do i have to first load the text to xml and do the transformation? is there any other way to do it?

8. problem while working with fixed length strings in Vb.Net - Microsoft .NET Framework