Jump to content

HAIF upgrade - September, 2022


editor

Recommended Posts

Well that took a metric ass-load of time. 

Sorry that HAIF was offline for... lemme check... 15 hours!

I moved everything to a new server to prepare for future upgrades.  Hopefully I managed to piece everything back together again and it all works.

If you notice anything strange, e-mail me at editor@houstonarchitecture.com, so i can get on it right away.  Hopefully I don't have to restore the backup.

As a point of interest, HAIF is currently 16.9GB in size. 

With the upgrade that finished a few minutes ago, there are some new features.  But it's almost midnight, and I'm tired, and the cat is acting up, so I'll post about them tomorrow.

  • Like 6
  • Thanks 1
Link to comment
Share on other sites

7 hours ago, editor said:

Well that took a metric ass-load of time. 

Sorry that HAIF was offline for... lemme check... 15 hours!

I moved everything to a new server to prepare for future upgrades.  Hopefully I managed to piece everything back together again and it all works.

If you notice anything strange, e-mail me at editor@houstonarchitecture.com, so i can get on it right away.  Hopefully I don't have to restore the backup.

As a point of interest, HAIF is currently 16.9GB in size. 

With the upgrade that finished a few minutes ago, there are some new features.  But it's almost midnight, and I'm tired, and the cat is acting up, so I'll post about them tomorrow.

Your hard work is much appreciated.  I'm sure I'm not alone in this sentiment.

Even though the IT guy in me knew this was probably just some extended maintenance (based on the timing), in the back of my mind I starting thinking about the chances the site might go offline permanently at some point.

I may have brought this up before and just don't remember, but is there a plan to pass the db on to a local institution if it ever has to shut down permanently?  The reason I say this is there is quite a bit of oral history encompassed here, both contemporary to events (see the Heights Walmart thread) and from people who post their recollections on Houston of old.  This site is a unique resource for local history that doesn't exist anywhere else and would be valuable in the UH or HPL collections if it goes dark online.

Just my two cents...

  • Like 3
Link to comment
Share on other sites

3 hours ago, august948 said:

is there a plan to pass the db on to a local institution if it ever has to shut down permanently

I've never thought about it.  It seems like a good idea.

I recently shut down the Chicago Architecture web sites, which contained thousands of articles and thousands of facts, blueprints, and other content available nowhere else.  An organization there approached me about archiving it somewhere, but then stopped responding to e-mails.  It's a shame, because I didn't feed it all into archive.org because I thought the org would be a better steward of it.   Now it's all sitting on a hard drive bit-rotting.

It would be a shame for the same thing to happen to HAIF.  I don't know that any local institution would be interested.  But I think I'll make an effort to ensure that all of the pages of HAIF are at least submitted to archive.org.  It's not much, but it's something.

As for the upgrade, there were two reasons it took so long.  First, I moved HAIF to a new sever, which is never fun.  And second, the move required rebuilding the message database, which is over 600,000 items.  Plus indicies of all of the uploaded files, and a million other things.

I kept popping in to look at the progress bar slowly inch along:  0.03% complete... 0.11% complete... 1.02% complete...

Link to comment
Share on other sites

3 minutes ago, editor said:

I've never thought about it.  It seems like a good idea.

I recently shut down the Chicago Architecture web sites, which contained thousands of articles and thousands of facts, blueprints, and other content available nowhere else.  An organization there approached me about archiving it somewhere, but then stopped responding to e-mails.  It's a shame, because I didn't feed it all into archive.org because I thought the org would be a better steward of it.   Now it's all sitting on a hard drive bit-rotting.

It would be a shame for the same thing to happen to HAIF.  I don't know that any local institution would be interested.  But I think I'll make an effort to ensure that all of the pages of HAIF are at least submitted to archive.org.  It's not much, but it's something.

As for the upgrade, there were two reasons it took so long.  First, I moved HAIF to a new sever, which is never fun.  And second, the move required rebuilding the message database, which is over 600,000 items.  Plus indicies of all of the uploaded files, and a million other things.

I kept popping in to look at the progress bar slowly inch along:  0.03% complete... 0.11% complete... 1.02% complete...

I feel your pain.  I can't even count the number of times I've done upgrades, server moves, full environment restores, etc. over the course of my career, some taking multiple days to complete.

There's a chance someone on this board is in a position at UH, particularly in the school of architecture, to archive this or has someone in their rolodex who might be interested.  If so, speak up.  It'd be a shame to lose this as I'm sure it's also a shame that Chicago has lost similar info.

  • Thanks 1
Link to comment
Share on other sites

So, you may have noticed that HAIF was unavailable for the last mumble mumble mumble  hours.  Sorry about that.

I had to upgrade the server again.  The new operating system was larger than the old one.  The new HAIF software indexes a lot more information.  In short, we ran out of storage and I had to upgrade the drive.

So now, we have a really big, expensive hard drive, twice as many CPU cores, and twice as much memory as a few days ago.  That will hopefully keep everything running for the next five years or so.

Sorry about that.

  • Like 8
Link to comment
Share on other sites

Earlier I stated that HAIF is 17GB in size. That is incorrect. It is 163.74 GB.

16 hours ago, cspwal said:

Is it a physical server in a closet? Or all AWS or similar?

It's a VPS.  Here's the details of the upgrade:

- The HAIF server is now at 5851 West Side Avenue, North Bergen, New Jersey, with a lovely view of the Manhattan skyline.  Why New Jersey, and not Houston?  New Jersey has fewer natural disasters than Houston, and is closer to both Houston and major internet connections than the other options I liked (London, Berlin, Singapore, Seattle, etc.).

Screen Shot 2022-09-06 at 8.14.27 AM.png

- The new server now has eight virtual CPUs, 16GB of memory, and 320GB of storage.

- Switched from Google's reCAPTCHA to hCaptcha.  

- Improved spammer detection.

- Upgraded operating system to Ubuntu 20.04LTS

- For the nerds in the audience, HAIF now has an IPV6 address.  

- Search bar has been simplified.

 

Coming soon (and by "soon" I mean "eventually and maybe"):

- The ability to see that someone else is replying to a thread at the same time you're writing a reply to a thread.

- The ability to search for objects in photos.  I'm not sure I'll enable this until I better understand the implications, privacy and otherwise.

- Possibly replace "Popular Now" with "Trending Now," which is somehow different.

  • Like 7
Link to comment
Share on other sites

  • 5 months later...
On 2/17/2023 at 9:28 PM, Urbannizer said:

Why does auto-embed no longer work? One use to be able to C+P a link from a social media post and it’d automatically format itself to show in your comment. This feature has been out for some time. 

300 Colorado

Auto-embed seems to be working on my end.  I just pasted a link from your Flickr account, and I can see it fine.  Can you see the image above?

On 2/18/2023 at 2:11 AM, IntheKnowHouston said:

You can no longer edit your own comment or title of the topic you created after a certain time limit. Seems like this happened in the last week or so. Is it possible to restore the ability to edit our comments and topics?

The amount of time allowed to edit a comment has been reduced from ∞ to 30 minutes.

The reason for this is that spammers have learned to take over dormant accounts, then go in and alter previously posted comments to fill them with spam.

  • Like 1
Link to comment
Share on other sites

7 hours ago, editor said:

The amount of time allowed to edit a comment has been reduced from ∞ to 30 minutes.

The reason for this is that spammers have learned to take over dormant accounts, then go in and alter previously posted comments to fill them with spam.



Thank you for the response. The reduced time is unfortunate, but I understand.

Link to comment
Share on other sites

On 2/19/2023 at 5:18 PM, editor said:

300 Colorado

Auto-embed seems to be working on my end.  I just pasted a link from your Flickr account, and I can see it fine.  Can you see the image above?

The amount of time allowed to edit a comment has been reduced from ∞ to 30 minutes.

The reason for this is that spammers have learned to take over dormant accounts, then go in and alter previously posted comments to fill them with spam.

I mean for social media accounts like Instagram, Facebook, etc. You use to be able to paste the link into a reply and it’d automatically embed that post from those sites. Anyone remember this?

Link to comment
Share on other sites

On 2/22/2023 at 12:34 PM, Urbannizer said:

I mean for social media accounts like Instagram, Facebook, etc. You use to be able to paste the link into a reply and it’d automatically embed that post from those sites. Anyone remember this?

Ah.  Yeah, that's a problem.  And not one that I can solve.

The Facebook/Instagram embeds happen through a magic service that links HAIF with Meta.  It has to be set up on the HAIF Facebook page in order to work.  Which is was.

However, around 2018, Facebook locked me out of my account.  I haven't been able to get back in.  I've gone through all the hoops on its web site, and even sent in pictures of my gub'mint ID twice, but received no response. 

A few months ago, Facebook sent me an automated e-mail stating that it is upgrading its magic linking service to a new version, and the old one would stop working.  I was unable to upgrade the link because I still can't log in to my Facebook account, no matter how many hoops I jumped through.  My guess is that Facebook has finally turned off the old service now, in favor of the new one.  Thus, the link between no longer works.

It's an important lesson for everyone:

  • You get what you pay for.  Facebook costs $0, and thus you get $0 worth of customer service.
  • Do not tie yourself to any of the big data platforms.  Facebook, Google, and others are notorious for kicking people off of their platforms at random, often for no reason.  Have you ever used "Sign in with Google" or something similar?  Think about what would happen if you suddenly got all of your personal and business Google accounts suspended because your kid said a naughty word into a microphone while playing a video game.  Crap like that happens all the time.

I'm sorry I don't have better news for you.  But since I don't live in the E.U., I have no rights and no remedy when big tech companies misbehave.

The dumb thing is Facebook requiring the use of that linking service for something as simple as a post embed.  As you saw above, Flickr allows embeds without going through a bunch of crazy hoops. Even newly-evil Twitter allows it.  I just embedded a tweet below.  It's just Meta that is being stupid.

 

  • Like 2
Link to comment
Share on other sites

I looked into this a bit more last night, and I'm not right.

The connection thing that expired is only used for the "Log in with Facebook" function, and not for embedded content. 

I'll look into it a bit more to figure out why embeds are not working.

In case you're curious, these are the services that embeds are supposed to work from:

  • College Humor,
  • Coub
  • Dailymotion
  • Facebook
  • Flickr
  • Funny Or Die
  • Gfycat
  • Google Maps
  • Hulu
  • Instagram
  • ReverbNation
  • Screencast.com
  • SmugMug
  • SoundCloud
  • Spotify
  • TED
  • Twitter
  • Ustream
  • Vimeo
  • Vine
  • Wistia
  • YouTube
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...