Learning to Walk, Again.

So, lately, I’ve been realizing just how crippling a dependence upon the expediency of using the Esri suite of GIS software really is.  In my position as a GIS Specialist and Instruction coordinator at an Ivy League school, it’s been easy to let myself just design every solution based upon the availability of some “out-of-the-box” solution from Esri.  I put “Out-of-the-Box” in quotes here, for a reason.  For years now I’ve felt like punching every Esri employee I hear say that phrase during a demonstration.  It’s nothing personal, it’s just that the phrase is disingenuous, at best.  Unless you are willing to compromise on the requirements of a particular project, whether it is field data collection, data distribution, analysis, metadata creation… whatever, there is really no such thing as an “Out-of-the-Box” solution, because there is no such thing as an “Out-of-the-Box” problem.

Ok, if all you need to do is put your vacation photos in Google Earth, or something trivial like that, there are a few “Out-of-the-Box” solutions.  But the projects that come across my desk are rarely that straightforward.

Take, for example, a project that has been making my life miserable, periodically, for a few years.  It would seem fairly straightforward to survey street trees in a city, right?  But I’ve been struggling to create a decent solution for just such a survey, since 2007. The trouble has been that there has never been a single “Out-of-the-Box” solution to all of the requirements of the project, so that the last 6 years have been one compromise after another to meet the needs.  In a nutshell, the project has these requirements:

  1. Multi-user editability (10-20 users)
  2. Disconnected (our University won’t pay for cellular service) editing.
  3. Cached custom basemaps
  4. Preserving historical records of each tree.
  5. Immediate access to data in various client applications (web, desktop, etc…)

Now, call me crazy, but you would think those wouldn’t be all that difficult to build a solution for, right?  Well, try doing it with Esri’s “Out-of-the-Box” solutions.  The obvious solution to the need to preserve historical data was to create a relational database, with LOCATIONS separated from the data on the state of the location (presence of a live tree, dead tree, stump, species, DBH, etc…) in a separate, but related INSPECTION table.  Obvious, right?  Well, I’ve been listening to Esri Dev’s promise and fail to deliver a working solution to related table editing in the field for all of the 6 years I’ve been working on the project.  Even the one solution (ArcPad) that PROMISED this kind of functionality never worked properly and simply choked under the strain of our 36k locations and 75k+ Inspection records, plus a custom basemap.  Last summer at the ESRIUC 2012, I asked when the much touted ArcGIS Online for Orgs and it’s mobile applications for iPhone/iPad/Android would be supporting related table editing, and was told, “Oh, we’re working on that and it should be out by October.” The crickets are still chirping on that one.

I’m still at a loss as to WHY Esri still hasn’t addressed this GAPING HOLE in all of the supposed “Field Data Collection Solutions.”  Have these people ever collected data in the field?  Do they really NOT see the value of a TRULY RELATIONAL DATABASE MODEL?  The only answer I can come up with is “NO” given the anecdotal evidence above, as well as their piss-poor implementation of SQL and relational database mechanics in ArcGIS platforms of all flavors (Desktop/Server/Mobile, etc…).

And, the Esri solutions have just become so overbloated and unnecessarily complex that it’s nearly impossible to make real use of them.  I co-taught a directed study with Dr. Dana Tomlin this semester for a student who was interested in learning more about deploying the type of Enterprise GIS and Data Collection solutions that ArcGIS Server and the new ArcGIS Online for Organizations have been promising.  Now, I’ve implemented production SQL Server>SDE>ArcGIS Server tech stacks about a dozen times  over the last 6 years for various projects, but for the first 3 weeks of this poor kid’s directed study, we were completely lost in the new SDE installation procedures, etc… Thank Imhof for the ESRI Forums, where the masses go to wonder what the f*$k is keeping their “Out-of-the-Box” solutions from working correctly, and try to help each other work things out.  Buried about 20 replies into an obscure thread we finally found the answer… turns out THE ARCGIS SERVER STACK DOESN’T NEED SDE ANY LONGER!  Now, at the end of the semester, we finally have a working prototype of the system, with exactly ONE feature class and feature service AND A DAMNED 40 PAGE DOCUMENT EXPLAINING THE TECH STACK!  That, is ridiculous.

So that brings me to the title of this post:  “Learning to Walk, Again.”  I started using ArcINFO as a command-line Unix program, back in 1996 at the University of North Texas.  In 1997, I took my first workshop to learn how the new GUI-based ArcView worked.  I wish I’d never left the Command Prompt.  Really, the GUI is just a crutch.  Does it make things faster? Probably not.  Does it make it easier to learn the software? Maybe.  What it DOES do, though, is move you further and further from what makes computers so powerful for solving problems, and that is knowing how to manipulate code and creating solutions for problems rather than fitting the problems to available solutions.  So, I am embarking upon a rigorous schedule of mental therapy, this summer.  One that I hope will “get me back on my feet” again and allow me to shed what has become, more and more, a crippling addiction to Esri’s proprietary GUI-based “Out-of-the-Box” solutions.  Unfortunately, I am only now realizing that there is no such thing as an “Out-of-the-Box” problem.

Look for more posts in the coming days and weeks, about the journey.  Tag along, if you like.  I’m going to start at the beginning and work my way back up.

Baby Steps.

Using your Own Tilecached Basemap in ArcPad

So, the ArcPad Dev Team has been promising tilecached basemap data support in ArcPad for a while, now.  I’ve been exited about this because we use a high resolution basemap (including sidewalk edges, building footprints, etc…) in the ArcPad applications we use for mapping Street Trees with the Urban Resource Initiative at Yale.  We’ve been using the navmap basemap technique which improved performance, a little, but we’re still talking vector data for an entire city so the performance boost was slight, at best.

How to Switch Your Sirf-based GPS from Sirf to NMEA Protocol

This post applies only to the Juno ST, and ONLY TO THE ONES I AM WORKING WITH!  I can’t make any guarantees about it’s effectiveness for anyone else, but the steps outlined have resolved serious GPS communication bugs, for the units I am working with.

I’ve been having a hell of a time getting ArcPad to communicate properly with my Trimble Juno ST units since version 8.  I think I have finally figured out what the problem is, though.  It seems that, when you run Trimble’s GPSCorrect software on a Trimble Juno ST (and I suspect any Trimble unit with a Sirf chip), the software sets a switch on the Sirf chip to output Sirf Protocol, only.

My problems started when I did a clean reset of Windows Mobile to update from ArcPad 7 to ArcPad 8.  At that point, our version of GPSCorrect was not supported by ArcPad 8, so I didn’t reinstall it, thinking I would just use the NMEA Output for GPS communication.  This is where the little hardware switch issue I noted above started spitting Gremlins into the Junos.  It seems the GPSCorrect software left the Sirf Chip outputing Sirf Protocol data streams.

Now, you might think “no problem, I’ll just set ArcPad to use Sirf,” right?  Well, that’s what I thought, too.  WIth the Sirf chip outputting Sirf Protocol data, I had problems with GPS communication in ArcPad whether it was set to NMEA or Sirf.  These problems included indefinite “No Fix”; obtaining a fix, then “freezing” on the initial location and never updating; or just flat out crashing ArcPad.

So here is what I did to resolve the issue:

  1. Download and place SirfTech.exe on the Juno Units [http://w5.nuinternet.com/s660100031/SirfTech.htm]
  2. Use File Explorer to browse to and launch SirfTech.exe
  3. Open the COM Menu and set the COM Port to 7
  4. Click on the Find Baud button and wait for the program to find the Baud rate that the Sirf Chip is passing data at. (On the Juno ST Units I have the problems with, this is always 38400)
  5. Once it finds the Baud Rate, you should see the Messages updating for the SiRF Protocol, but not NMEA. Click OK.
  6. On the Menu go to Sirf>Switch to NMEA Protocol.
  7. Set the Baud Rate to 4800 and click on the Set button.  Click OK.
  8. Return to the COM page and Close the COM Port.
  9. Change the Baud Rate to 4800 and click Open to reopen the COM Port using NMEA.  You should see the Messages updating for the NMEA Protocol, now.
  10. Finally, Close the port, click OK and Exit the SirfTech program

You should now be able toSet the GPS Preferences in ArcMap to NMEA 4800 and successfully Fix and Update your GPS Position.