Learning to Walk, Again.

So, lately, I’ve been realizing just how crippling a dependence upon the expediency of using the Esri suite of GIS software really is.  In my position as a GIS Specialist and Instruction coordinator at an Ivy League school, it’s been easy to let myself just design every solution based upon the availability of some “out-of-the-box” solution from Esri.  I put “Out-of-the-Box” in quotes here, for a reason.  For years now I’ve felt like punching every Esri employee I hear say that phrase during a demonstration.  It’s nothing personal, it’s just that the phrase is disingenuous, at best.  Unless you are willing to compromise on the requirements of a particular project, whether it is field data collection, data distribution, analysis, metadata creation… whatever, there is really no such thing as an “Out-of-the-Box” solution, because there is no such thing as an “Out-of-the-Box” problem.

Ok, if all you need to do is put your vacation photos in Google Earth, or something trivial like that, there are a few “Out-of-the-Box” solutions.  But the projects that come across my desk are rarely that straightforward.

Take, for example, a project that has been making my life miserable, periodically, for a few years.  It would seem fairly straightforward to survey street trees in a city, right?  But I’ve been struggling to create a decent solution for just such a survey, since 2007. The trouble has been that there has never been a single “Out-of-the-Box” solution to all of the requirements of the project, so that the last 6 years have been one compromise after another to meet the needs.  In a nutshell, the project has these requirements:

  1. Multi-user editability (10-20 users)
  2. Disconnected (our University won’t pay for cellular service) editing.
  3. Cached custom basemaps
  4. Preserving historical records of each tree.
  5. Immediate access to data in various client applications (web, desktop, etc…)

Now, call me crazy, but you would think those wouldn’t be all that difficult to build a solution for, right?  Well, try doing it with Esri’s “Out-of-the-Box” solutions.  The obvious solution to the need to preserve historical data was to create a relational database, with LOCATIONS separated from the data on the state of the location (presence of a live tree, dead tree, stump, species, DBH, etc…) in a separate, but related INSPECTION table.  Obvious, right?  Well, I’ve been listening to Esri Dev’s promise and fail to deliver a working solution to related table editing in the field for all of the 6 years I’ve been working on the project.  Even the one solution (ArcPad) that PROMISED this kind of functionality never worked properly and simply choked under the strain of our 36k locations and 75k+ Inspection records, plus a custom basemap.  Last summer at the ESRIUC 2012, I asked when the much touted ArcGIS Online for Orgs and it’s mobile applications for iPhone/iPad/Android would be supporting related table editing, and was told, “Oh, we’re working on that and it should be out by October.” The crickets are still chirping on that one.

I’m still at a loss as to WHY Esri still hasn’t addressed this GAPING HOLE in all of the supposed “Field Data Collection Solutions.”  Have these people ever collected data in the field?  Do they really NOT see the value of a TRULY RELATIONAL DATABASE MODEL?  The only answer I can come up with is “NO” given the anecdotal evidence above, as well as their piss-poor implementation of SQL and relational database mechanics in ArcGIS platforms of all flavors (Desktop/Server/Mobile, etc…).

And, the Esri solutions have just become so overbloated and unnecessarily complex that it’s nearly impossible to make real use of them.  I co-taught a directed study with Dr. Dana Tomlin this semester for a student who was interested in learning more about deploying the type of Enterprise GIS and Data Collection solutions that ArcGIS Server and the new ArcGIS Online for Organizations have been promising.  Now, I’ve implemented production SQL Server>SDE>ArcGIS Server tech stacks about a dozen times  over the last 6 years for various projects, but for the first 3 weeks of this poor kid’s directed study, we were completely lost in the new SDE installation procedures, etc… Thank Imhof for the ESRI Forums, where the masses go to wonder what the f*$k is keeping their “Out-of-the-Box” solutions from working correctly, and try to help each other work things out.  Buried about 20 replies into an obscure thread we finally found the answer… turns out THE ARCGIS SERVER STACK DOESN’T NEED SDE ANY LONGER!  Now, at the end of the semester, we finally have a working prototype of the system, with exactly ONE feature class and feature service AND A DAMNED 40 PAGE DOCUMENT EXPLAINING THE TECH STACK!  That, is ridiculous.

So that brings me to the title of this post:  “Learning to Walk, Again.”  I started using ArcINFO as a command-line Unix program, back in 1996 at the University of North Texas.  In 1997, I took my first workshop to learn how the new GUI-based ArcView worked.  I wish I’d never left the Command Prompt.  Really, the GUI is just a crutch.  Does it make things faster? Probably not.  Does it make it easier to learn the software? Maybe.  What it DOES do, though, is move you further and further from what makes computers so powerful for solving problems, and that is knowing how to manipulate code and creating solutions for problems rather than fitting the problems to available solutions.  So, I am embarking upon a rigorous schedule of mental therapy, this summer.  One that I hope will “get me back on my feet” again and allow me to shed what has become, more and more, a crippling addiction to Esri’s proprietary GUI-based “Out-of-the-Box” solutions.  Unfortunately, I am only now realizing that there is no such thing as an “Out-of-the-Box” problem.

Look for more posts in the coming days and weeks, about the journey.  Tag along, if you like.  I’m going to start at the beginning and work my way back up.

Baby Steps.