Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

Read sql data from a large file

 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> General
View previous topic :: View next topic  
Author Message
satya61229
Smarty Rookie


Joined: 11 Nov 2006
Posts: 8
Location: India

PostPosted: Wed Nov 22, 2006 7:46 am    Post subject: Read sql data from a large file Reply with quote

Suppose you have to write a script for reading insert data from a very large file (8-9 Mb).
Then What you would do?
Generally Script for this read all data in variable and then read one line a time and insert.
But in this case reading 8MB data in memory itself jam the PHP.

So i have written a script after not getting in google.

I read a chunk of data from file.
Then i read each line.
Check the data in line. Suppose you data is in CSV form and all quoted with double quote. then you can count the (") in line.
You can also extract the line and check that you have right no. of column in that line.
If data in line is not correct then it will be you last line in you read chunk.
So go back the bytes you read. and start with new chunk including this bad data in bytes you read.
You have to make account of data you already read, data to read, chunk size..
I did it for reading MaxMain data, Which is in CSV form.

And make sure, you will run insert query at the last of every chunk read.
and only this way- Insert into tablename (col1, col2..) values (val1,val2), (val1,val2), (val1,val2) ,,,.

satyaprakasha1@yahoo.co.in
http://satya-prakash.blogspot.com/


Last edited by satya61229 on Wed Nov 22, 2006 10:26 am; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
shannera
Administrator


Joined: 13 Feb 2006
Posts: 802
Location: Edertal, Germany

PostPosted: Wed Nov 22, 2006 8:21 am    Post subject: Reply with quote

So what it is you want us to tell about nice smarty tips and tricks?
Back to top
View user's profile Send private message
satya61229
Smarty Rookie


Joined: 11 Nov 2006
Posts: 8
Location: India

PostPosted: Wed Nov 22, 2006 10:27 am    Post subject: Reply with quote

shannera wrote:
So what it is you want us to tell about nice smarty tips and tricks?


Sorry! I thought it is for any kind of tips and tricks related to programming.
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Hielke Hoeve
Smarty Elite


Joined: 06 Jan 2006
Posts: 406
Location: Netherlands

PostPosted: Wed Nov 22, 2006 12:56 pm    Post subject: Reply with quote

All forums are for questions/answers have to be related to Smarty.
_________________
Debug XHTML Compliance
SmartyPaginate
Smarty License Questions
---
(About Unix) The learning curve is full of aha! moments, such as that glorious day that the full beauty of grep and, later, find is revealed in all its majesty. --- Robert Uhl <ruhl@4dv.net>
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Nov 22, 2006 3:22 pm    Post subject: Reply with quote

...and now moved to the General forum, though still off-topic Wink
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP