Mass Wasting Videos

Dalhousie, northern India 2011

Calabria, Italy, 16 Feb 2010

View Larger Map

Debris Flow: Clear Creek County, Colorado, Spring 2003

Liquefaction in New Zealand

Rock Slide in Tenn.

Japan 2004

Malaysia Tin Mine Disaster – slumping example

 


 

Oso Washington, 2014

Why iCloud Does Not Mean You Can Stop Backing Up!

I was excited about iCloud.  I suppose I still am, to a degree.  However, I quickly figured out that you cannot count on iCloud as a backup solution for your apps and music.  Why?

I downloaded the beta iTunes 10.3 and began checking around.  In terms of music, I have somewhere around 700 purchases that are DRM’d alone.  However, right now the total number of songs available for me on iCloud is around 200.  If I assume they’re still building the database, then that might be okay.

In terms of apps, I have a little more than 320.  As far as I can tell, the iCloud database is complete as it will get, but not all of my apps are there.

What’s the lesson that I’m taking from this?

If the item is not available on the store, it will not be available on iCloud!

Yes, that’s right, many things I’ve purchased on the iTunes store are no longer available.  Hence, it’s not on iCloud.  Whether this will be also true for any post-iCloud purchases, I don’t know.  But it’s seems clear to me at this point that anything not currently in the store will not be available on iCloud.

For example, I purchased 4 King Crimson songs from the album THRAK!.  They are DRM’d acc files.  After I made that purchase, King Crimson objected to being on the store and were removed.  I still have my purchases, though.  I fully do not expect these song to be available on iCloud because the songs are still not available.

In the app store, Delicious Library was removed after my purchase.  In this case, it was removed for a complex set of reasons I won’t get into here.  However, it’s not on the store, and it’s not in iCloud.

So, what to do?  I will be quite difficult to figure out what’s on iCloud and what’s not.  Apps are split into iPhone/iPad lists and it’s impossible to figure out the exact count of apps because of the way it’s split.  I’m not sure how many people will take the time to figure out what’s available and what’s not.

So, iCloud is promising, but it has serious holes, or at least lack of clarity, in the area of the availability of past and future purchases.  Being able to download the files again is great news, for at least those files that are available.  But I don’t think most people will keep track of what you purchases and is not available.  Just back up everything.  While the iCloud logo looks a bit heavenly, it is not a safe solution for your purchases…

 

A genealogical gold mine?

Over the last year or so, I’ve been interested my family’s genealogy.  It’s been quite the adventure.  Most of my mother’s side of the family has been done for some time, but my father’s side was largely unknown (and to a large extent, still unknown).   Much of my successes come from the relatively recent availability of online data, particularly census and marital records.

However, one data set has largely been unavailable – regional history.  Historical events are important in genealogy because it places your family in context with what was going on at the time.  Where there battles near where they lived?  Did they participate in events?  Did those events contribute to their struggles?  One of my goals I have with my genealogical records is not only to incorporate the data into GIS and animations, but also bring in historical data for context in maps.  I want to see how history impacted my family through time.  However, collecting historical data is hard.  Even if you have the time, determining what’s important and where the events occurred would be extremely difficult, particularly for several hundred years of history.

Interestingly, this problem may have been solved.  A BBC software engineer named Gareth Lloyd produced a unique database by accessing Wikipedia.  The database was compiled from Wikipedia pages that contained latitude and longitudes and dates.  Using this database, Mr. Lloyd produce a short animation of the historical event in map form (see here).   The database is now available via Google Fusion.  What this may mean is that it is now possible to integrate in some sort of GIS environment genealogical data AND historical data, at least those historical events that made it to Wikipedia.

Two drawbacks currently exist with these data, however.  First, I haven’t actually downloaded the data and put it into a GIS yet.  Second, I haven’t derived an approach to do the same with my genealogical data either. Thus, right now, this is still a pipe-dream.  Third, it’s unclear that these data will be updated regularly.  It looks like this project was extremely large to assemble and wouldn’t be easy for someone with bandwidth caps to replicate.    Updating these data would be important over the years, but I don’t see a clear approach for this right now.

In the end, if the practical  problems can be worked out, then I think these data are very important for genealogical research.  We’ll have to wait see what happens.

 

 

Why I sometimes really hate linux-y stuff

I’m one of those people always on the fence when it comes to open source software.  Sure, the idea is great, but when it comes to using it… well, I’m often disappointed.

Case in point: I’m trying to install GnuCash on my mac(s).  There are installations that are easy, but they come relatively limited, particularly in terms of file formats.  GnuCash can support nice database systems like mysql or postgres rather than just the original XML format.  I want that capability.  So, I try to compile the code required to add that support.  No joy.  Grrr.  So, I decided to go with the MacPorts install…  hours ago… I’m still waiting for all of the compiles of dependencies.  The process so far has stopped 3 times on failures.  It’s rather ridiculous.  I’m sure someone would say “If you just used linux it wouldn’t be a problem”.  Sure, you’re likely to be successful installing and probably more quickly on linux, but I don’t like linux very much.  I’ll use it when I have to.

So, when I complain about linux, I’ll have to remember to add the whole issue of dependencies.  I usually gripe about things like UI, which most people see right away.  However, the dependencies issue turns me off very quickly.   Let’s face it, the dependencies issue won’t go away.  You could argue that’s the linux philosophy, but I suspect it’s really the Gnu OSS licensing making this stuff crazy.  Let’s face it, open source software really isn’t free.  Most OSS software  comes with restrictions.  I’m often willing to abide by these restrictions as a user because I really don’t have to think about them.  However, if I understand the licensing (and I freely admit I don’t understand the licensing that well) there is a strong disincentive to compile  and distribute the software, statically linked to it’s dependencies, as one big app.  Thus, you’re stuck with installing dependencies that might or might not be compatible with the application you’re really trying to build.  So, part of the cost of OSS is dealing with these kinds of issues.

Despite this complaining, the cost of OSS (and it’s more than just dependencies) is sometimes worth it.  Some OSS software I use often are QuantumGIS and Zotero (I probably use far more than that, particularly in terms of libraries).  I’m hoping that GnuCash fits in this category of being “worth it”.

 

What I’m hoping with GnuCash is that I have sort of an in-house common financial record system.  Right now, I use GnuCash for my business accounting, but I’m wanting to shift away from Quicken for our personal work.  We have a really old version of Quicken and I can’t justify in my mind the cost of upgrading, particularly since it doesn’t do the job the way I need to use it.  In particular, it seems unsafe to use a Quicken file across a network connection.  That always scares me with Quicken.  My hope is that getting GnuCash working with Postgres or Mysql that sort of thing would be safer.   I understand it’s still not safe for two people to use at the same time, but that’s okay.  I want a safer and robust approach that my wife might be willing to use.

 

ah finally, it’s done.

UPDATE

Bah, humbug.  All that work for nothing.  Apparently, my installed version of Postgres is incompatible with libdbi.  Geez!

Behind on accounting: Speeding it up with python and sqlite

Well, as usual, I’m behind on last years accounting and tax time is rapidly approaching.  I finally sat down and at least sorted my receipts saved from last year.  Now I’m faced with the horrible data entry phase.

Now, I was able to download all the transaction records from the bank.  That’s all well and good, but it doesn’t have any of the information I really require except the amounts.  Even the payees are primarily transaction number that are barely readable.

As usual, I decided to see what’s out there and available to help with taking a QIF file and make it useful.  I found one or two things for the Mac, but they were pricey for what they do.  However, I suddenly remembered that last year I made a solution to do exactly what I needed (How could I have forgotten??).

If you poke around the net, you can find a python script to read and write QIF files (thanks to whoever wrote that!).   Thus, that is part one of the solution, the ability to read and write QIF files.

Part 2 of the solution is SQLite.  Sqlite is a great low-overhead SQL database system.  Works great with python.  In sqlite, then you can build a list of expected transaction information.  My table is fairly simple:

Name: the name of the transaction as I’d like it to appear

qifname: the partial name of the transaction as it appears in my QIF file.  For example, Netflix appears as “WWW.NETFLI”

Category: the optional assignment of a category if it’s fairly consistent.

Part 3 is a custom python script that incorporates the QIF read and writes, checks the database for each transaction against the SQLite database for the qifname, if it’s found, replaces the transaction name with name and addes the category when present.

Simple, free, and powerful.  Hours of work saved.

 

 

Drobo Drive Failure: Lesson Learned? Again?

Well, my Drobo is up and running again.  All my data appears in tact.  Wonderful!  Now we go merrily along like nothing happened, right? Not quite.

Today, I have the backup bug.  The backup bug is a wonderful thing.  Simply put, I have the incredible urge to back everything up.  Make multiple copies of everything I don’t want to lose and securely distribute those copies out into the universe where, one day, I might call on them to resurrect themselves.  Yeah!

I also know  something about the backup bug: it will go away quickly.  Today my data are safe, and they’ll probably be safe tomorrow.  I can put it off for a couple days while I resolve other priorities.  Days will become weeks and weeks will become an unknown amount of time until my next hard drive failure.  It will come.

When that hard drive failure comes again, and it will, I’ll be kicking myself, fixing the problem and hopefully moving on with minimal data damage, but with a huge loss of time.  While I probably will have minimal loses, it’s still a waste.  With 2 TB hard drives getting down to as little as $100 these days, it’s cheaper to buy a bunch and consistently backup rather than lose a couple days fixing the problem.  It’s a hard lesson to learn.

What really kills off my backup bug is organization.  I have the space to properly backup, particularly today.  However, the question because what and how to backup.  I don’t want to backup everything, just those things that are not adequately backed up.  I have several categories for my data:

1. Active  – Active data are those things I’m actively modifying.  Writing projects, for example.  These require constant backup.

2. Inactive – Formerly active projects that are retired.  I may want them on my computer or server, but they only need multiple static copies.

3. Configuration – those files that are used to configure my computer – the OS, for example.  These, like Active projects, should be constantly backed up but not necessarily in the same place.

4. Replaceable – if the files are not really mine, like an application disk image, and can be reacquired generally shouldn’t be backed up, unless they are no longer available.

5. Temporary – judgement call – once it’s backed up, it might be around for a really long time.

In any case, none of my files are organized this well… I should probably get to it if I’m going to get anything before the backup bug goes away.

Another drive failure in my Drobo

Drive failures are a fact of life.  We just have to get used to it and be prepared.  I have a stack of failed drives in my basement, destined for destruction and eventual recycling.  Losing a drive is never happy, is often painful, and is quite disruptive.

One of the most difficult things in computing, particularly home computing, is being prepared for drive (or computer) failures. Let’s face it: it’s expensive and hard.  There are those that wisely suggest the 3-2-1 approach to data protection: 3 copies of everything, 2 media formats (like a hard drive and a DVD), and 1 copy offsite.  Who can really do that? Well, most people really.  For a typical computer user, you probably have less than 500 gb of data, and probably far less “important” data.  With that amount of data, you could easily satisfy the 3 copies and 1 offsite requirement.   The 2nd media format could also include online storage in the cloud (yes, that counts as offsite too!), which is getting more reasonable all the time.

For me, however, I have tons of data.  In fact, I have so much data that the 3-2-1 approach is particularly difficult for me to meet.  In fact, I’m building a database just to track hard drives!  I probably have somewhere around 9-20 terabytes of drive space floating about.  The question is “How much of that is sufficiently backed up?”  Probably too little of it.  I’d say less than 1% meets the 3-2-1 criteria.  However, I’d say have at least half at the 2-1-1 level.

For what doesn’t even meet the 2-1-1 level, I have my Drobo.  A Drobo is a bit like a RAID, but requires less knowledge to run and is a bit more flexible.  When it’s in good condition, it protects your data from a drive failure.  For me, it serves 3 basic rolls: “temporary” storage of active projects, “archival” storage for things I have backups elsewhere, and the “pit of despair” for all the data that came to either die and be forgotten or data I just never got around to backing up.  The problem, of course, is that the Drobo somewhat tricks you to think that you don’t need to back up your data just yet. You still have to back it up.  Do as I say! (not as I do)

This past Sunday, when trying to get busy, my Drobo Dashboard app started flashing an angry Red flash.  Drive failure.  Remember, all is okay with my data, the Drobo protected it.  However, now all the data are in an unsafe state – the Drobo can no longer ensure the safety of my data until I replace the failed drive.   When drives fail, my goal is simply to buy a larger capacity drive and replace it.  Unfortunately, this failed drive was the largest in my Drobo.  Gak!  So, no upgrade is likely to happen.   Worse, I’m not in a position to run out and get a new drive – I had to order one and the shipment is delayed because of weather.  Gak!

So, I’m offloading data from my Drobo.  Right now, since the largest drive failed, I’m really REALLY over the safe capacity of the Drobo, so I need to get the overall storage level down to meet the new limitations.  That’s about 1 terabyte of data I need to offload.  Gak!   Arguably, I don’t have to do this at all.  Instead I could just wait for the drive to arrive and plop it in.  However, the Drobo’s performance is really really bad in this state.  Furthermore, all those “temporary” and “pit of despair” files are all at risk.  Sigh.

I should point out that I think this might actually be my 3rd drive failure in my Drobo.  I don’t think that’s unusual, after all, over the years I’ve had many many drives fail.  After all, I run them pretty hard.  It just reinforces the need for backups and redundant drive solutions.

Once everything is good, it’ll be time to check on those warrantees.

My first thoughts on the Second Generation AppleTV

My AppleTV arrived a while ago and I thought it might be time to share initial reactions.

My first reaction is that it was a mighty breath of fresh air in our TV room.  We dropped cable tv a few weeks ago and moved an old G4 mac into our TV room to support iTunes content.  That combination was fine, but the big and load G4 was more of an annoyance.  The mess of cables, keyboard, mouse, UPS, etc was just something get in the way.  Getting all of that out of the TV room was a big improvement.  Instead of a huge noisy box with a mess of cables, we now have a tiny box with just a power cord and HDMI cable.

The AppleTV is relatively easy to use.  It has a simple interface that even my 3 year old can use.  It also small enough to move around to other rooms, should you want to watch content on another TV and don’t want to buy another box.  So far, the watching experience using iTunes, Netflix, and YouTube has been quite good.

One thing I haven’t yet tried (and need to try) is set the Parental Controls.  We failed to use Parental Controls on Comcast’s system and that cost us $5 in an unwatched movie (thanks to our 3 year old).  Fortunately, right now it’s impossible to rent (or buy) anything since I haven’t set up the accounts.

The overall interface, however, is clearly geared to the iTunes store to rent content.  I don’t know how much of that I plan to do in the future (it’s reasonable to rent in some cases), so it’s slightly annoying that so much of the interface is based on content I’m not going to access very often.  Hence, parental controls is a must if I enter any iTunes account information.

The only real problem with the device I’ve noticed so far is hangs trying to access remote iTunes libraries.  Since the AppleTV has no hard drive, it’s critical to have good networking to the machines that are hosting the source iTunes libraries.  It appears, at times, it has a lot of trouble connecting to libraries and only pulling the plug on the device helped.  However, it’s unclear whether this is a problem with the device, with iTunes, or with my home network (non-apple 802.11n set up).  It could also be that the computers go to sleep and the connects do not update properly.  Either way, it can be annoying.

An indirect problem with this set up is the inability to set ratings on music and video content in iTunes.  I would love to be able to tag rating to all my content which would allow me to specifically allow or disallow what my kids can access in my iTunes library.  Without that ability, I find I have to have, at a minimum, 2 iTunes libraries: a master library, and a kids library.  The kids library contains just what I think is okay for them to watch/listen.  Right now, that means I have libraries set up on two different computers (although, come to think of it, it might be possible for me to do it on the same computer, which would be better since only one of the libraries would be available at a time).

The most surprising thing about the AppleTV is how much more I play music in the TV room.  We did have a Generation 1 iPod connected to the stereo, but we never used it.  We forgot what was on it, didn’t want to bother managing it, etc.  With the AppleTV, we have full access to all the music – which actually serves as a really good way to get the TV off and the kids playing.

Why I am getting an AppleTV

I’ve been wanting to get an AppleTV for years, but I’ve never pulled the trigger until now.  The machines were always reported as hot (and I recollect reports of noise).  Worse, it probably wouldn’t have worked on our TV.  The final nail was that it was too expensive for the risk of not doing the job we wanted.

An  AppleTV is mainly a way to view your iTunes content on your TV.  Up until the new version, the AppleTV was a hard-drive based system that synced content to you main iTunes library on your computer.  The new (and much cheaper) version dumps the syncing and the hard drive and adds a few more outside connections like Netflix.  Furthermore, the new version is tiny and very quiet.

So, what changed my mind?

First, we decided to drop Cable TV.  We  spend way too much money on TV.  Given the shows we watch, we realized that most of what we want to see is on broadcast TV, Hulu, netflix, or other web sites.  The shows that aren’t freely available on the net are often easily purchased while still saving money over the cost of  Cable TV.  Thus, if we choose to buy, we would have an avenue to play the shows both on the computer and on TV (Amazon doesn’t work with macs, AFAIK, and thus gets ruled out).

Second, we made the move to HDTV.  It’s not much of a move; we sold our existing TV and are using my HD-capable computer monitor as an HDTV.  Interestingly, the size of this TV (24-inch) really shows the fact that 1080p is really only useful at 50 inches or large and that when we eventually upgrade (between 32-40 inches), 720p is enough.

Third, we have invested in the iTunes universe.

Forth, I want to minimize out-of-the-home networking.  Other boxes exist that can handle iTunes content, but understanding is the content is  pushed to outside servers and then to the box.  My internet provide does cap the amount of bandwidth I can use in a month (breaking that cap can lead to disconnection).  Thus, I don’t want any bandwidth wasted on that sort of thing.

Fifth, the AppleTV is not a computer.  When we made the decision to drop Cable TV, we reluctantly moved a computer in our family room to play iTunes content.  It is impossible to use as a computer, it’s really noisy, and takes way way too much power and space for the room.  We want it gone ASAP.

Sixth, the AppleTV is really small and has few cables.  There are many reasons this is good.  But a surprising one is that the simpler the set up, the easier it is to move into other rooms.  Eventually, we want a small HDTV in another room so easily moving the box would be great.

The AppleTV for use is a “No Brainer”.  The cost is relatively low risk if we decide we don’t like it (anything over $100 is getting up there, and anything over $200 is flat out of the picture – it’s just friggin TV after all).  The announcement of the new version shifted our focus from buying a blu-ray player (after all, the size of the TV we have wouldn’t do blu-ray justice anyway) and allows us to consider the possibility of replacing our TV with something a little larger.

I’ll report more once the device arrives.

Retro Movie Review: Rollerball

Rollerball started life as a short story by William Harrison called Roller Ball Murder.  This story, published around 1973 became the basis of the first Rollerball movie in 1975.  Rollball holds a special place of personal nostalgia because it was one of the first movie for adults that I remember watching (R-rated) and one of the first big scifi movies I remember seeing (there were other excellent movies before Rollerball, but I had yet to see them).

Rollerball is a strange story.  It’s set in the future where corporations have become governments and those corporations used sport to foster both loyalty to the corporation and, in the case of rollerball, prove that the individual is not important, only the group.  Rollerball accomplishes the latter because the game is designed to be extremely violent and players, then, would typically have a very short career.  However, Jonathan E. didn’t  fit the mold.

Jonathan E. survived and excelled at rollerball.  By surviving, he put the corporations in an awkward position.  They couldn’t just sack him; he was too popular.  So, they told him to retire.  However, the game was all he had left in life so he refused.  Thus the story become the conflict between the corporations that want him gone because he disrupted the purpose of the game and Jonathan, a relatively simple man, simply did not want to quit.

I’ve heard mixed reviews of James Caan’s portrayal of Jonathan E.  I think he played it fairly well since I think Jonathan’s character was that of an uncomplicated person struggling in a life where everything around him is full of schemes and machinations.  He’s a classic reluctant hero.

Overall, I still enjoy this film.  I like it’s setting (which turns out to be filmed in Germany), the music (classical score), and stark contrast of the incredible violence of the rollerball games and the  “intellectual”  settings almost everywhere else while all of it being a bit insane.

In contrast, Rollerball 2002 was a disappointment.  It is totally possible to make a better movie than the original, but it just didn’t happen in this case (despite the star-heavy cast).  Instead of a futuristic (and arguably unrealistic) world of the original movie, this movie is set former Soviet  Union and the game itself is a centerpiece of business.  The game itself in hard to figure out – why are they going through that tunnel?  The overall plot seems more like a story about human trafficking – most players are poorly paid, and once they’re in they can’t get out. A few scenes in the movie appeared to emulate scenes from the original, but they simply didn’t have the power of the original.  I really didn’t care about anyone in the film, about the setting, or the game.