Author Topic: tropical.theferns.info down?  (Read 1727 times)

elouicious

  • Hero Member
  • *****
  • Posts: 1355
    • Houston, Tx
    • View Profile
tropical.theferns.info down?
« on: December 08, 2022, 10:19:03 PM »
Hey all-

this website is an amazing resource that many of us use but I am not sure how many people are aware that it is run by a small husband and wife team- the Ferns

Is it down for everyone or just me? I would hate to see this resource lost, but even when trying to navigate to the donation page it seems to be down

sytanta

  • Sr. Member
  • ****
  • Posts: 286
    • Ho Chi Minh, Vietnam
    • View Profile
Re: tropical.theferns.info down?
« Reply #1 on: December 09, 2022, 01:46:13 AM »
This has happened several times to me too. Not sure if we can do some donation/gofundme for the website's team.

tongmuan

  • Member
  • ***
  • Posts: 28
    • Thailand
    • View Profile
Re: tropical.theferns.info down?
« Reply #2 on: December 09, 2022, 02:48:13 AM »
Theferns server has returned only empty pages for a few weeks already, but the Useful Tropical Plants site can still be browsed at least partially via the Wayback Machine.

I wanted to post a direct link to the archive, but "CleanTalk" (whatever it is) assumes I'm a spammer and doesn't let me post links here...

To get to the Useful Tropical Plants archive:
1. Type web.archive.org in the web browser url bar
2. Enter tropical.theferns.info in the web archive url field
3. Click "November 1, 2022." (or whatever is the latest date displayed) that appears below the url field
4. You are now in the latest archived front page

tongmuan

  • Member
  • ***
  • Posts: 28
    • Thailand
    • View Profile
Re: tropical.theferns.info down?
« Reply #3 on: December 09, 2022, 02:57:33 AM »
Here's a clickable link to the currently latest archived front page:
https://web.archive.org/web/20221101154350/https://tropical.theferns.info/


W.

  • Hero Member
  • *****
  • Posts: 647
    • United States, Alabama, 7b
    • View Profile
Re: tropical.theferns.info down?
« Reply #4 on: December 09, 2022, 05:37:14 PM »
It's been down before. I noticed that a large number of new plant entries appeared on the site after the last time they were down. I'm also not getting any sort of alert about the website being unavailable; I just get a blank screen when I go to their homepage. I'm hoping they're just doing another website update and will be back up very soon. I agree that it is an excellent resource and one that we are very lucky to have.

Vegan Potato Man

  • Member
  • ***
  • Posts: 160
    • Honaunau, HI
    • View Profile
Re: tropical.theferns.info down?
« Reply #5 on: December 09, 2022, 08:02:07 PM »
Its been down before for extended periods. Not sure what the problem is but I cant load it now either.

Many of the posts there are duplicated on "plants for a future", FWIW

pfaf.org

Hosting my own websites, I know it can be pretty complicated.

I've had more issues with my email server though. We use healthchecks.io to provide alerts if postfix goes down.
« Last Edit: December 09, 2022, 08:03:46 PM by Vegan Potato Man »

culi

  • Newbie
  • *
  • Posts: 5
    • California coastal sage and chaparral
    • View Profile
    • culi.page
Re: tropical.theferns.info down?
« Reply #6 on: January 07, 2023, 02:09:22 AM »
pfaf is not nearly as extensive for tropical species as tropical.theferns.info is

I think the latter is actually a fork so it already includes all pfaf data in it. There's also temperate.theferns.info which is also down

I've never seen either of them down for this long and to be honest it's really worrying. Does anybody know of any possible way to contact Ken Fern or someone close to them? I know there's discs of the data floating around but I'm not sure if that's just the original PFAF data or if it includes the latest version of the tropical db

Unfortunately the Wayback Machine only has a handful of the pages archived :(

Next time it comes back up I'm definitely going to try to make sure it's archived and might even scrape it (the robots.txt doesn't seem to forbid it)

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #7 on: January 07, 2023, 02:21:37 AM »
I swear one day last week I clicked on a .theferns link and it WORKED!

Like said earlier, I think they have just taken it down for maintenance to add more species; I could be hallucinating but I swear it worked for me one day waaay after this initial outage post
instagram @trumansacco

culi

  • Newbie
  • *
  • Posts: 5
    • California coastal sage and chaparral
    • View Profile
    • culi.page
Re: tropical.theferns.info down?
« Reply #8 on: January 07, 2023, 04:39:23 AM »
I swear one day last week I clicked on a .theferns link and it WORKED!

Like said earlier, I think they have just taken it down for maintenance to add more species; I could be hallucinating but I swear it worked for me one day waaay after this initial outage post

Yes it did in fact work just a few days after this post, but then it went down again around mid/late December. This time seems to be different though. The pages return an HTML but with no context. Just the styling (hence why you get a greyish background and not a 404 error). Even stranger is that the apex site is up: https://theferns.info/

But if you try to navigate to any of the 3 subdomains (tropical, temperate, and plantsforafuture) you get basically the issue described above Well the last one at least gives a proper 500 error. 500 means it's an actual server issue. Unlike 404 (meaning no content found at that url), 500 actually tells us something is messed up with the server logic itself. So we know the server is up, it's just broken somehow...

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #9 on: January 07, 2023, 08:10:15 AM »
I swear one day last week I clicked on a .theferns link and it WORKED!

Like said earlier, I think they have just taken it down for maintenance to add more species; I could be hallucinating but I swear it worked for me one day waaay after this initial outage post

Yes it did in fact work just a few days after this post, but then it went down again around mid/late December. This time seems to be different though. The pages return an HTML but with no context. Just the styling (hence why you get a greyish background and not a 404 error). Even stranger is that the apex site is up: https://theferns.info/

But if you try to navigate to any of the 3 subdomains (tropical, temperate, and plantsforafuture) you get basically the issue described above Well the last one at least gives a proper 500 error. 500 means it's an actual server issue. Unlike 404 (meaning no content found at that url), 500 actually tells us something is messed up with the server logic itself. So we know the server is up, it's just broken somehow...

just checked my history; I was able to open .theferns for chrysophyllum imperiale on 12/24 Don't lose hope!

« Last Edit: January 07, 2023, 08:13:16 AM by tru »
instagram @trumansacco

elouicious

  • Hero Member
  • *****
  • Posts: 1355
    • Houston, Tx
    • View Profile
Re: tropical.theferns.info down?
« Reply #10 on: January 07, 2023, 10:34:41 AM »
its was down for me yesterday

culi

  • Newbie
  • *
  • Posts: 5
    • California coastal sage and chaparral
    • View Profile
    • culi.page
Re: tropical.theferns.info down?
« Reply #11 on: January 10, 2023, 05:35:13 PM »
It's back!  ;D

https://tropical.theferns.info/

Still says 12,727 species so I don't think they added any new ones. Has anyone been able to get in touch with them? I tried emailing them to offer technical help but haven't gotten a reply (part of why I began to fear the worst)

I wish I could buy their database

cassowary

  • Hero Member
  • *****
  • Posts: 606
    • Australia FNQ 13a Tropical Monsoon
    • View Profile
    • cassowaryseeds
Re: tropical.theferns.info down?
« Reply #12 on: January 12, 2023, 06:06:24 PM »
There will be a way to have a software print all pages into pdf files and name them with the species name, that way you could have a private database of it. I don’t think it would be evil to do so if it is just for your own eduvational purpouses.
It would give only one image, since it would print only whats on the species landing page.

This way there wouldn’t be any need to get inside access or have to download the xml data itself and rearrange it in a list.
CASSOWARYSEEDS.COM
Seed shop and Seed exchange

elouicious

  • Hero Member
  • *****
  • Posts: 1355
    • Houston, Tx
    • View Profile
Re: tropical.theferns.info down?
« Reply #13 on: January 12, 2023, 06:29:24 PM »
There will be a way to have a software print all pages into pdf files and name them with the species name, that way you could have a private database of it. I don’t think it would be evil to do so if it is just for your own eduvational purpouses.
It would give only one image, since it would print only whats on the species landing page.

This way there wouldn’t be any need to get inside access or have to download the xml data itself and rearrange it in a list.

I made a half assed attempt at this once that I may need to revive

cassowary

  • Hero Member
  • *****
  • Posts: 606
    • Australia FNQ 13a Tropical Monsoon
    • View Profile
    • cassowaryseeds
Re: tropical.theferns.info down?
« Reply #14 on: January 14, 2023, 12:42:56 AM »
Elouicious,
Did you find some software or did you try to code something?
Any advice?
CASSOWARYSEEDS.COM
Seed shop and Seed exchange

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #15 on: January 14, 2023, 02:14:58 AM »
hey I tried my hand at it,

scraping the data itself is really difficult from the way they coded it, the text is just in a random <div> with a bunch of citation links that cause web scrapers to mess up

If you just want a list of the URLs, I grabbed all of them PM me and I'll share excel sheet if interested, If you're comfortable with code there's probably tutorials for quickly getting it done, my theory coding would be:

step 1) read in an iteration from CSV file
step 2) convert a given link to whatever file you're going for
step 3) make code to save the name the file based on the link given
« Last Edit: January 14, 2023, 02:20:15 AM by tru »
instagram @trumansacco

culi

  • Newbie
  • *
  • Posts: 5
    • California coastal sage and chaparral
    • View Profile
    • culi.page
Re: tropical.theferns.info down?
« Reply #16 on: January 14, 2023, 01:24:49 PM »
I've also scraped it. The HTML is really messily arranged. My plan was to parse it to grab all the data and format it as a CSV (for spreadsheets) or JSON collection (for APIs), but I ended up just downloading all the HTML files to have locally for now. So I basically have the website saved locally. This way when I try to clean it up and parse the site I don't have to make a bunch of calls to their server anymore

I'll probably convert it to CSVs in a little bit, but for now I'm just relieved to have it backed up

EDIT: PS, fwiw, I do donate to the site to keep it running

cassowary

  • Hero Member
  • *****
  • Posts: 606
    • Australia FNQ 13a Tropical Monsoon
    • View Profile
    • cassowaryseeds
Re: tropical.theferns.info down?
« Reply #17 on: January 14, 2023, 06:13:14 PM »
I found this open source software:
https://www.httrack.com/page/2/

"t allows you to download a World Wide Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer."

Does not get detected as malware and testing it now.

tropical.theferns.info
Is Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
CASSOWARYSEEDS.COM
Seed shop and Seed exchange

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #18 on: January 14, 2023, 06:33:32 PM »
sounds good, I think something like this https://www.web2pdfconvert.com/ would work just fine in a pinch

culi, I was wracking my brain lastnight trying to figure how to go about fixing my issue, are you getting the same function () { .. .. . . . } calls when you scrape for text? If you are, I figured out an interesting solution!

use RightToLeft in RegexOptions and match for a string that starts with } and ends with 'function'. I don't know if that's what you're getting too but that cleared up a lot of my problem
instagram @trumansacco

mbmango

  • Member
  • ***
  • Posts: 194
    • LAX, CA
    • View Profile
Re: tropical.theferns.info down?
« Reply #19 on: January 15, 2023, 10:17:10 PM »
Scraped into a google sheet: https://docs.google.com/spreadsheets/d/1QpB1tqf5kXakDSqJhtYSYhu-DbDmcpiY-yegjmcxPDA
Just make a copy if you want to fiddle with it or the backing apps script for scraping.  When opening your own copy, you should see a "Scrape" menu to kick off scraping, but it won't overwrite anything.  You'll have to delete any plant or reference you want to get refetched.

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #20 on: January 16, 2023, 12:24:41 AM »
beautiful, thanks so much. Gonna go mess around with this in PowerBI for a couple hours  ;D
instagram @trumansacco

culi

  • Newbie
  • *
  • Posts: 5
    • California coastal sage and chaparral
    • View Profile
    • culi.page
Re: tropical.theferns.info down?
« Reply #21 on: January 16, 2023, 01:06:26 AM »
culi, I was wracking my brain lastnight trying to figure how to go about fixing my issue, are you getting the same function () { .. .. . . . } calls when you scrape for text? If you are, I figured out an interesting solution!

Ah, that's because the javascript is in-lined. I use an HTML processing library to help with this. Right now I'm using Ruby which has a really nice gem called "nokogiri" that handles situations like this nicely. If you're scripting with Javascript then it actually has DOMParser built in. Python also has HTMLParser. All of these allow you to use CSS query selectors to navigate the html much easier and more predictably

elouicious

  • Hero Member
  • *****
  • Posts: 1355
    • Houston, Tx
    • View Profile
Re: tropical.theferns.info down?
« Reply #22 on: January 17, 2023, 02:41:57 PM »
so glad to see our CS people stepping up!

so huge-

thanks guys for this effort-

Has anyone had luck contacting Ken?

Just hoping all is well with them

tru

  • Sr. Member
  • ****
  • Posts: 471
    • Texas
    • View Profile
Re: tropical.theferns.info down?
« Reply #23 on: January 17, 2023, 04:37:30 PM »
hey been playing around with formatting and have this so far, definitely not done but should paint a pretty good picture, what do yall think? Thanks so much to everyone that came together to make this happen

Filter with the first tab, or explore the information the site normally shows on the next page, 3rd page is expandable if you click on the bars. 4th I havent got to start yet. Click on latin names to filter, pretty much just click on stuff and see if its interactive

https://app.powerbi.com/view?r=eyJrIjoiM2E3ZDc5MDYtNDIzZi00NzgxLTlhNmItNjI5NDEyZDUxZDk0IiwidCI6ImNhODU2YzQ5LTFkNTQtNGYzMS04ODEzLWFiMTJmZGNmZGQ1MSJ9&pageName=ReportSection
« Last Edit: January 17, 2023, 04:44:20 PM by tru »
instagram @trumansacco

elouicious

  • Hero Member
  • *****
  • Posts: 1355
    • Houston, Tx
    • View Profile
Re: tropical.theferns.info down?
« Reply #24 on: January 17, 2023, 05:42:22 PM »
this is really dope tru-

how much will it cost to host?