Python Data Science and the Rocket MultiValue Database ( Part 3 of 3 )
Now that we have gone over some of the basics such as manipulating numpy arrays and pandas data frames in the first two parts of “Python Data Science and the Rocket MultiValue Database”, we will now create a simple object for managing the storage and retrieval of data to a Rocket MultiValue database.
For our object we will need to import the following:
import u2py import numpy as np import pandas as pd import json
Note that we will also be using the same file we created in part 2 of this blog series. ( If you do not have a U2DS file in your account, please go back to the other post to get the command to create it. Click here if you want to read part 1 of this series.)
For the remainder of this blog post, I will be using the mvdf.py module which I created and attached to this post.
python> from mvdf import *
While we could have created individual methods to handle saving my nested python lists, numpy arrays, and pandas data frames to a Rocket MultiValue database, this module provides an object that will allow you to store, retrieve and marshal any of the data into the type you require.
Let’s start by instantiating an mvdf object:
python> test = mvdf() python> test.to_list() ['', '', '']
Our object has three properties: a comment, columnNames and an array. To start populating our object let’s set the comment and the array:
python> test.comment = "This is a test" python> test.array = [ [1,2], [3,4] ] python> test.to_list() ['This is a test', '', [[1, 2], [3, 4]]]
We can now marshal a numpy array, or a pandas data frame from our object.
python> npa = test.get_numpy_array() python> npa array([[1, 2], [3, 4]]) python> pddf = test.get_pandas_df() python> pddf 0 1 0 1 2 1 3 4
Since this is a sample object we have not implemented all aspects of constructing the pandas data frame, like data type and index, but we did add code for column names.
python> test.columnNames = [ "Field-1", "Field-2" ] python> pddf = test.get_pandas_df() python> pddf Field-1 Field-2 0 1 2 1 3 4
Since this is a post about “Python Data Science and the Rocket MultiValue Database” now we need to write our object to the Rocket MultiValue Database.
python> test.write() wrote the item to 9
And of course we would want to get our data back out of the Rocket MultiValue Database.
python> new = mvdf(9) python> new.to_list() ['This is a test', ['Field-1', 'Field-2'], [['1', '2'], ['3', '4']]]
Please look at the attached code, and feel free to modify it to meet your needs. The initial code stores your nested lists, numpy arrays and pandas data frames to the Rocket MultiValue database, then retrieves the data, and marshals into the needed form.
I hope this gets you started in the fun world of “Python Data Science and the Rocket MultiValue Database.”