UTF8 in Crowbar

by lih on 29 January 2011 - 11:12pm in

Crowbar is an awesome scraping tool, but it messes up if the page contains non ascii characters. After snooping around crowbar's source, the following is a "fix" I came up with.

var converter = Components.classes["@mozilla.org/intl/converter-output-stream;1"].
  createInstance(Components.interfaces.nsIConverterOutputStream);
converter.init(outstream, "UTF-8", 0, 0);
converter.writeString(response);
//outstream.write(response, response.length); //<-----original

Awesome!

Thank you so much for your fix regarding UTF-8 and Crowbar!