Other

UTF8 in Crowbar

Crowbar is an awesome scraping tool, but it messes up if the page contains non ascii characters. After snooping around crowbar's source, the following is a "fix" I came up with.

var converter = Components.classes["@mozilla.org/intl/converter-output-stream;1"].
  createInstance(Components.interfaces.nsIConverterOutputStream);
converter.init(outstream, "UTF-8", 0, 0);
converter.writeString(response);
//outstream.write(response, response.length); //<-----original

Syndicate content