Just want to check if anyone has good ideas for storing large amounts of information that is read from an outside source (csv file).
I'm currently using individual lists to store the data.
I access the data by element, and this keeps the information locked together.
For instance, everything in row 4 needs to go together (each row is an individual activity).
If I wanted to store this in a class or object, would I still be able to keep the data "linked".
string studentName; //This information is repeated many times in the csv file, but it only needs to be stored once per student.
string courseName =new string; //The course name will also be repeated many times, however, a student may take 3 or 4 courses, each with up to 40 lessons.
string lessonName = new string;//A lesson may be taken once or more if the student fails, each attempt needs to be recorded alongside other data
string grade = new grade;// Each grade needs to apply specifically to 1 lesson
string date = new date;//Each date needs to apply specifically to 1 lesson
This is a basic outline, I have a lot more information to load.
My question is this,
Is it worth it to change my current system which is working fine to try to fine tune it into a class, or will it not be worth the extra work involved changing my code over?
If I'm dealing with hundreds of thousands of pieces of information, is using lists and having redundancy going to significantly slow performance of the program?
Thanks for any help and ideas!
Best Practice - Storing information
Post a Reply
|Oldest Newest Rating|
· October 1, 2014
Thanks for the response Eugene. The main reason I'm using a csv is because our database is offsite. I could use web APIs to grab the data directly (and plan on doing this in the future) but I preferred to have a working desktop application that I can send to our instructors asap. I'll do some tests with different classes and see how accurately I can grab the data and keep it logically correct.
· October 1, 2014
This is always a tricky situation. There isn't a single best answer. That's why it is generally best to keep the data in the DB rather than a csv. It is much easier to query and parse the info into usable classes.
What makes it so tricky, is that you have a kind of cyclic inheritance. For example:
Each student is enrolled in 3 or 4 courses.
Each course contains up to 40 lessons
Each lesson has students.
Each student is enrolled in courses //now we get back to the top, and the cycle can continue if not carefully implemented
When fetching the data from a DB, it is easy to bind the data for the kind of query you need. For example, if you want to report on the pass rate of a course, you start at the course, get the lessons, get the students and stop. If you want to query the pass rate of a student, you get the student, get the course, get the lessons and stop. From a csv, this isn't completely as easy.
My suggestion would to create a class wherever possible.
I'd say create a lesson class, a course class with an array of lessons, and a student class with an array of courses. The lesson class might need some tweaking if you plan of querying lessons/courses for all students.
Bottom line is: When you have data that is grouped, it's never a bad idea to create a class to keep the data together. It usually handles easier as well.