Minimal Entity Coding to Permit Feeds Import in Drupal 7

Drupal is great for managing content, by design!  But, when you start to amass a lot of content, getting into the range of “medium” data (let’s call that about 200k records), Drupal’s standard “node” storage mechanisms can start to cramp your style, especially if your content is fairly complex, i.e. having many fields.  One of the great benefits of the Entity API and module, is the ability to store some information in a more efficient manner.  This is because when using nodes, if my data has multiple fields, Drupal creates a separate table for each field, with several fields on that table therebey increasing storage requirements, as well as complicating the resulting queries in Views since a table has to be joined for every field used.  When using entity, I can create more traditional, multi-fielded database tables that can be dealt with much more economically.

For my example, I wanted a database table with 5 columns for storing numerical time series data that I could populate using the Feeds importer capabilities of the Feeds module.   A few handy resources made this possible:

  • Feeds importer was recently patched to be able to handle generic Entity types.  Feeds 7.x-2.x-dev contains the ability to automatically configure a “Processor” for any entity defined using the Entity module.  This was the result of a really long development effort by a lot of Drupalites: https://www.drupal.org/node/1033202
  • A nice tutorial was assembled to handle a “simple” case of Entity implementation, but that case proved a little more complicated than what I needed.
  • The Model example project provides templates as well, with very good information, but alas, still too complicated for me.

I wanted to:

  •  Have an entity defined by a module that would create my time series data table upon enabling the module.
  • Automatically have a Feeds Entity Processor appear in the Feeds Admin UI screens that would populate my table from an uploaded csv file.
  • Use the absolute minimum amount of settings in order to better understand the basics of Entities while still achieving my goals.

I managed to achieve my goals with such a minimum amount of coding that it speaks really powerfully about the package of capabilities provided by Entity (as well as the Feeds upgrades!).  My example, which used only 3 classes and 2 functions is described  as follows, with the relevant code and test datasets at the end of this blog.

Classes Used

  • Only 3 classes were used: EntityEntityDefaultUIController, and EntityAPIControllerExportable
  • All 3 classes were simple children of the parent class, with no extensions or sub-classing of methods.

Methods or “Hooks” Defined

  • hook_schema() – this is where I defined the database table that would be created upon install.
  • hook_entity_info() – this told Drupal some of the basics, such as what my class names were and some basic details about my Entity such as whether it was “fieldable” (it was not), what database table to use (defined in the install call to hook_schema()), and some basic plumbing.

Sample Files

File: drupal_hydro.admin.inc
I put all of my classes in this file.

<?php

/**
* UI controller.
*/
class DrupalHydroTimeSeriesType extends Entity {

}

class DrupalHydroUIController extends EntityDefaultUIController {

}

class DrupalHydroTimeSeriesTypeController extends EntityAPIControllerExportable {

}

?>

File: drupal_hydro.info
This contains the basic module declaration info with a line to tell it to include any extra coding files.  Drupal will always look for “[module name].install” and “[module name].module”, but some others need to be included in the files[] statements shown below:

name = DrupalHydro Module
description = Implementation of the ArcHydro Framework in Drupal
core = 7.x
php = 5.3.0

dependencies[] = entity
dependencies[] = views

files[] = drupal_hydro.admin.inc

 

File: drupal_hydro.install
The database table definition is here.  Note that this will create the database table for you.  It will also automatically append whatever Drupal database table prefix you are using, if any, so if you go manually looking in your db for you table, keep that in mind.

<?php

function drupal_hydro_schema() {
//
$schema[‘drupal_hydro_tstype’] = array (
‘description’ => ‘Data Dictionary for Variables in drupal_hydro (ArcHydro: TSTYPE)’,
‘fields’ => array(
‘tstypeid’ => array(
‘description’ => ‘Primary key for Variables’,
‘type’ => ‘serial’,
‘unsigned’ => TRUE,
‘not null’ => TRUE,
),
‘variable’ => array(
‘description’ => ‘Variable Name’,
‘type’ => ‘varchar’,
‘length’ => 128,
‘not null’ => TRUE,
),
‘units’ => array(
‘description’ => ‘Unit of Measure’,
‘type’ => ‘varchar’,
‘length’ => 32,
‘not null’ => TRUE,
‘default’ => ”,
),
‘isregular’ => array(
‘description’ => ‘Is Regular?’,
‘type’ => ‘int’,
‘size’ => ‘tiny’,
‘not null’ => TRUE,
‘default’ => 1,
),
‘tsinterval’ => array(
‘description’ => ‘Interval Type’,
‘type’ => ‘int’,
‘not null’ => TRUE,
‘default’ => 1,
‘size’ => ‘tiny’,
‘not null’ => TRUE,
),
‘datatype’ => array(
‘description’ => ‘Data Type (ArcHydro: TSDataType)’,
‘type’ => ‘int’,
‘not null’ => TRUE,
‘default’ => 1,
‘size’ => ‘tiny’,
‘not null’ => TRUE,
),
‘origin’ => array(
‘description’ => ‘Origin (ArcHydro: TSOrigins (1 = Recorded, 2 = Generated)’,
‘type’ => ‘int’,
‘not null’ => TRUE,
‘default’ => 1,
‘size’ => ‘tiny’,
‘not null’ => TRUE,
)
),
‘primary key’ => array(‘tstypeid’),
‘indexes’ => array(
‘dh_tst_tstix’ => array(‘tstypeid’),
‘dh_tst_vix’ => array(‘variable’)
)
);

//drupal_hydro_create_default_views();

return $schema;
}

function drupal_hydro_create_default_views() {
//$q = ” “;
//$result = db_query($q);
}

?>

File: drupal_hydro.module
This usually contains any fancy functionality that you want your module to have.  In this example, I didn’t want anything other than for it to be able to be accessed by the Feeds importers, thus the hook “hook info” was all that was needed.  This is based heavily on the example blog noted above, but since I did not want my table to be fieldable (at least not for now), I was able to skip the use of “bundle” declarations, leaving them empty.  If you do want to have them be fieldable, your database table needs to have a field on it that can store the bundle information to associate the extra field tables to.  In the “node” entity, and in the example above, they used “type” as the field that holds bundle information.  The “entity keys” array must have the name of this column defined as in ‘bundle’ => ‘[bundle field name’], as must the ‘bundle keys’ array.  Of course, make sure that your info array has “‘fieldable’ => FALSE” if you do NOT want to bundle.

<?php

/**
* Implement hook_entity_info().
*
* We define two entities here – the actual entity that will hold our domain
* specific information and an entity that holds information about the different
* types of entities. See here: http://drupal.org/node/977380 for a discussion on this
* choice.
*/
function drupal_hydro_entity_info() {
$return[‘drupal_hydro_tstype’] = array(
‘label’ => t(‘DrupalHydro TimeSeriesType’),
// The entity class and controller class extend the classes provided by the
// Entity API
‘entity class’ => ‘DrupalHydroTimeSeriesType’,
‘controller class’ => ‘DrupalHydroTimeSeriesTypeController’,
‘base table’ => ‘drupal_hydro_tstype’,
‘fieldable’ => FALSE,
‘entity keys’ => array(
‘id’ => ‘tstypeid’,
),
// Bundles are defined by the model types below
‘bundles’ => array(),
// Bundle keys tell the FieldAPI how to extract information from the bundle objects
‘bundle keys’ => array(
),
‘label callback’ => ‘entity_class_label’,
‘uri callback’ => ‘entity_class_uri’,
‘access callback’ => ‘entity_access’,
‘module’ => ‘drupal_hydro’,
// The information below is used by the ModelUIController (which extends the EntityDefaultUIController)
‘admin ui’ => array(
‘path’ => ‘admin/content/drupalhydro’,
‘file’ => ‘drupal_hydro.admin.inc’,
‘controller class’ => ‘DrupalHydroUIController’,
‘menu wildcard’ => ‘%drupalhydro’,
),
);

return $return;
}

?>

File: test_data.csv (actually a tab-delimited file)

variable units tsinterval datatype
wd_mg_mon mg_mon 19 2
wd_gal_mon gal_mon 19 2
wd_mg_yr mg_yr 19 2
wd_gal_yr gal_yr 19 2

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s