winsock >> Idhttp.Get() - Extra bytes appended

by doesnotmatter » Sat, 09 Aug 2003 12:21:36 GMT


Hello,
I while ago i had asked about the idHTTP.Get method for resuming downloads like:

if (FileExists(FName)) and (Resume) then
Fs := TFileStream.Create(FName,fmOpenReadWrite)
else
Fs := TFileStream.Create(FName,fmCreate);

try
IdHTTP1.Port := 80; //EdtPort.Value;
IdHTTP1.ProtocolVersion := PV1_1;
if Resume then
begin
Fs.Seek(0, soFromEnd);
IdHTTP1.Request.ContentRangeStart := Fs.Position;
end;
IdHTTP1.Get(FUrl, Fs);
...etc....

The problem i see is that the downloaded file with the Resume option set to true is appended by a few extra bytes. Dowsnt idHTTP check the filesizes ? Do i have to manually chack the sizes of local and remote files before using this option ?
Thanks and regards.




winsock >> Idhttp.Get() - Extra bytes appended

by doesnotmatter » Sat, 09 Aug 2003 12:26:49 GMT



Also, in the events OnWork, on WOrkBegin, OnWorkEnd etc, i have a status caption:
'Transfer:'+IntToSTr(AWork)+ bytes of'+IntToSTr(AWorkMax)+'bytes';
The AWorkMax value is always 4096 bytes and the AWork value may not even reach the entire 4096 bytes before beginnin again. Am i missing anything here ?

Thanks and regards.





winsock >> Idhttp.Get() - Extra bytes appended

by Remy Lebeau (TeamB) » Sun, 10 Aug 2003 07:10:57 GMT






Please show your actual code.


Gambit





Idhttp.Get() - Extra bytes appended

by doesnotmatter » Sun, 10 Aug 2003 13:10:55 GMT


I know its quite easy to get the remote file size with a IdFtp component but how to do with idHTTP ? Will idhttp.Response.getContentLength give an accurate read of the remote filesize or is there another mechanism for this?
Thanks and Regards








Idhttp.Get() - Extra bytes appended

by Remy Lebeau (TeamB) » Sun, 10 Aug 2003 17:15:52 GMT






No reliably. You can try doing a Head() request for the remote file and
then test the ContentLength, but not all servers send ContentLength during a
HEAD, if at all.


Gambit




Idhttp.Get() - Extra bytes appended

by doesnotmatter » Mon, 11 Aug 2003 03:20:57 GMT


Well, Thanks to you for the answers. I do have a last question. As i mentioned earlier, on the onWork event of the IdHttp component, the aWorkCount is the progress in terms of a maximum of 4096 bytes(AworkMax). Isnt there a way to get the file in a continuous stream rather than in chunks of 4096 bytes ?
Thanks and regards









Idhttp.Get() - Extra bytes appended

by Remy Lebeau (TeamB) » Mon, 11 Aug 2003 04:53:20 GMT






No. It is already being transmitted as a continous stream, but it is being
read and stored in chunks for reasons regarding memory usage. Each read is
triggering its own OnWork event.


Gambit





Idhttp.Get() - Extra bytes appended

by DoesntMatter » Tue, 12 Aug 2003 04:30:46 GMT


Thanks very much for your help.
Best Regards







Similar Threads

1. Why am I getting extra bytes?

I must have lost my mind because I cannot for the life of me see why
this is happening.  I'm on linux 2.6, gcc compiler and code that is
too simple to be playing tricks like this.

The server reports:
Read 216809472 Bytes, Wrote 216809472 Bytes for 'longmetro05.raw'

The client reports:
Read 216809479 bytes, Wrote 216809479 bytes to 'longmetro05.raw'

Please point out how stupid I am and where the problem is.

Here is the client code:
				int nTotalRead = 0, nTotalWritten = 0;
				while(1) {
					int nRead = read(socketfd, buf, sizeof(buf));
					nTotalRead += nRead;
					if(nRead > 0) {
						int nWritten = write(fd, buf, nRead);
						//printf("%d - %d\n", startat, nWritten);
						nTotalWritten += nWritten;
						if(nWritten != nRead) {
							fprintf(stderr, "Error writing '%s'", argv[2]);
							exit(-3);
						}
					} else if(nRead == 0) {
						// Connection closed by other end
						close(fd);
						break;
					} else if(nRead < 0) {
						close(fd);
						perror("Error reading socket");
						exit(-4);
					}
				}

				printf("Read %d bytes, Wrote %d bytes to '%s'\n", nTotalRead,
nTotalWritten, argv[2]);

Here is the server code:

		int nTotalRead = 0, nTotalWritten=0;
		while(1) {
			int nRead = read(fd, bufr, sizeof(bufr));
			nTotalRead += nRead;
			if(nRead) {
				lastOffset = lseek(fd, 0, SEEK_CUR);
				int nWritten = write(pInfo->clientfd, bufr, nRead);
				nTotalWritten += nWritten;
				//printf("%d - %d\n", lastOffset, nWritten);
				while(nWritten != nRead) {
					fd_set writefds, exceptfds;
					struct timeval timeout = {5, 500000};

					FD_ZERO(&writefds); FD_ZERO(&exceptfds);
					FD_SET(pInfo->clientfd, &writefds); FD_SET(pInfo->clientfd,
&exceptfds);

					int nSelected = select(pInfo->clientfd+1, NULL, &writefds,
&exceptfds, &timeout);
					if(nSelected) {
						if(FD_ISSET(pInfo->clientfd, &writefds)) {
							int nWrote = write(pInfo->clientfd, &bufr[nWritten], nRead-
nWritten);
							//printf("%d %d\n", lastOffset, nWrote);
							nWritten += nWrote;
							nTotalWritten += nWrote;
						}
					} else {
						if(FD_ISSET(pInfo->clientfd, &exceptfds)) {
							close(fd);
							return -10;	// Exit thread
						}
					}
				}
			} else {
				  printf("Read %d Bytes, Wrote %d Bytes for '%s'\n", nTotalRead,
nTotalRead, cmd);
					//We are finished
					close(fd);
					return -10;
				}
			}
		}

2. extra 1 byte attachment added to multipart message

3. Extra bytes in packets

Hi!
I'm experiencing an anomaly on a certain linux machine. When I run
ethereal, it picks up packets fine but there is a string of extra bytes
at teh end of all packets except those sent by the host. It's a redhat 7
machine, I suspected the cersion of ethereal, so I upgraded the cersion
0.9.3 with the same results. Has anyone run into this problem? If so, I'd
appreciate advise.
Thanks in advance,
Allan
 XXXX@XXXXX.COM 

4. IdHttp uploaded files 2 bytes too large

5. getting a value from bytes in Pascal

I have a data format in Pascal and real values are 32 bits in this.
Bit 15 is a sign bit , zero indicates positive number.
Bits 14 through 7 are excess 128 binary component.
Bits 6 through 0 and 31 through 16 are a normalized 24 -bit fraction
with falling sinificance in that order. The most significant bit 6 is
worth 1/4.
I just read the 4 bytes with C++ in a Linux machine. Both pascal and
Linux have the same order for bits.
the individual bytes in floats are -57 17 66 58 ----> i want to find
the real value.
-57 in binary ------- 11000110 < bits 31 to 24
17 in binary ------- 00010001 < bits 23 to 16>
66 in binary ------- 01000010 < bits 15 to 8 >
58 in binary ------- 00111010 < bits 7 to 0 >


As per the above format bit 15 is 0. So sign is 0.
exponent is from from 14 to 7 ---- 10000100 = -4
so, excess exponent = 4;
and mantissa is
0111010 11000110 00010001 0 = 7703586


I am not sure how would I caluculate the number and also the weights.

6. getting a value from bytes in Pascal

7. Getting a bad byte in a stream

Hi, 

When reading through a winsocketstream I seem to be getting bad data
which is causing a huge problem in my application.  The data that is
being written out has been checked and it is being written to the
stream as it should be but when read back in the data is incorrect but
only after its read a substantial amount of the stream without
problem.  (i.e. I'm assuming its just one bad byte).

      for i:=1 to FEventCount do  //loops fine until about the 20th
iteration
        begin
          with FGroupEvents[i] do
          begin
            Read(nEventRef, SizeOf(Integer));
            Read(nDeviceId, SizeOf(Integer));
            Read(nDeviceType, SizeOf(Integer));
            Read(nSignalIdx, SizeOf(Integer));
            Read(bState, SizeOf(Boolean));
            if not (aStream is TFileStream) then
              EventState[i] := bState;

            Read(EventSeverity, SizeOf(TCosmosEventSeverity));
            nStrLength := 0;
            Read(nStrLength, SizeOf(Integer)); //This is eventually
reads a bad value which throws out the loop below
            sAssignedName := '';
            for j := 1 to nStrLength do
            begin
              ch := ' ';
              Read(ch, 1);
              sAssignedName := sAssignedName + ch;
            end;
          end;
        end;

8. Problem with Socket Server using IOCP getting 0 bytes under heavy