I m tired of seeing sidebars on my phone. It s great that my phone can handle every page on the web, and I can muddle through, but it s been four years since the iPhone killed the mobile web distinction and it s time for web design to catch up. I can t do anything about ugly design, but I can at least make it dead simple to show your mobile users a different ugly stylesheet. There are a lot of…
If you found this on Google and just want to know how to make text readable on iPhones and Android phones, set the text size to 24pt and call it a day. For the curious, read on. When I redid my website using Jekyll , one of my goals was to make sure that it was usable from mobile devices like an iPhone or Android. I was incredibly annoyed when I discovered that making a simple standards-compliant…
After a few days of work, I ve converted to Jekyll , a blog oriented static site generator. I never felt like learning enough about Movable Type to get it to do things I wanted (like a simple layout and maybe just accepting an HTML page without trying to interpret it), so I m very happy to be working with a set of files, in a set of directories, no less, that gets converted into a blog according…
People have difficulty internalizing what happens when you have a low error rate in some sort of detector but also have a very low probability event. I ran into this doing social media monitoring when we saw that the majority of items classified as certain uncommon language were wrong. Check out occurrences of "fuck" in English language books in 1862. Every single result on the front page is a…
Lately I ve been working on connectivity with NASDAQ. The protocols involve parsing fixed-offset messages of varios types. We re not doing high frequency trading so we are optimizing for programmer efficiency that is, the API I expose to the rest of the system should make sense, so I m representing the different types of messages, trading conditions, exchange identifiers and so on as enums. I was…
This post details Biz360 's automated sentiment analysis system, including our goals, how the system works, how we measure success, and the ways it can be used and misused. Before getting into the how or why , I want to start with the what . For our purposes, sentiment is the opinion of the author of an article towards the subject of an article. We classify sentiment into four possible categories.…
A coworker asked me today how to roll back a change that has been committed to SVN. This isn t obvious and the top google searches return irrelevant results. To back out or roll back a change that has already been committed to the Subversion repository, you first merge your commit in reverse, and then you commit. That is, in change 2918, you committed some config files that should not be there. Do…
We ve been running HBase at Biz360 for about six months, but it worked so smoothly at first that I never did much tuning. I ve recently increased the volume of data we re storing by about 300%, and have started running into some problems like blocks going missing. I found a good way to monitor all my servers logs to get an idea what s going on today, and it s so simple I wanted to share. for x in…
One item that stuck in my mind during a first aid class during boot camp was how to direct someone to call 911. Wrong: "Somebody call 911!" Right: "You, in the black shirt, go call 911." If something isn't the responsibility of a particular person, it will not get done. Everyone assumes someone else will do it. The same thing applies to software, and especially maintenance issues. If you say "hey,…
Hadoop Map Reduce and HDFS are fairly stable pieces of software. One component that doesn t have a clear winner yet is higher level job scheduling, also known as workflow scheduling. To put this in context for someone who isn t familiar with Hadoop, a single Hadoop job is broken up into many map and reduce tasks. The scheduler runs on the job tracker and assigns tasks to open slots on the task…
It s obvious to any rational outside observer that US terrorism policy mostly revolves around making sure people think politicians are doing something , regardless of whether something needs to be done, or whether what they re doing is the right thing. Explaining the work for which Williamson won the Nobel last week, Judge Posner writes: [FBI criminal-investigation functions] lend themselves to…
We ve run into several situations in Hadoop where we want to prevent a job from using more than a certain number of slots. Some of our jobs have external resources that don t scale. One task needs to talk to a MySQL database. Another writes to our Solr cluster. These are jobs that we know beyond a certain point they don t go any faster if we have 200 mappers running, it s not any faster than 50.…
HBase is primarily a sorted distributed hash map, but it does support secondary keys through a contrib package called Transactional HBase. The secondary keys are provided by a component called TableIndexed. A good general walkthrough is Secondary indexes in HBase on Rajeev Sharma s blog. My blog post specifically addresses how to use secondary indexes outside of the Java API and how to handle…
HBase includes an example for populating a table from Hadoop map reduce, but it seemed overly complicated. I m getting started with HBase and this was my starting point. This first one uses the old Hadoop API with everything in the mapred package, not mapreduce. It also uses the corresponding API from HBase which is now deprecated. public class PopulateArticlesTable extends Configured implements…
I m attending tonight a presentation on actors and actors in scala, presented by SDForum Software Architecture SIG. Upcoming meeting: July 21, 3rd tuesday, Vijay Patel linked in talk about analytics. Carl Hewitt, Stanford; Robey Pointer, Twitter; Frank Sommers, Artima; Bill Venners moderating. Abstract to concrete. I ll post this as-is and come back and edit it later. Carl Hewitt, Stanford,…
(Well dammit, I meant to save a draft, and here I ve accidentally published it and FriendFeed spreads it around like a gossipy neighbor, so I guess I d better finish it.) I ve been working on a collaborative filtering system based on genetic algorithms for message boards , running shoe recommendations , the Netflix Prize for a while now. The latest iteration is a mix of Java and Scala. I sat down…
Amazon Web Services offers two forms of data storage. First is S3, a key-value store allowing very large files if needed, but with a pricing model that will cause problems for small files. Second is SimpleDB, a schema-less or column-oriented database, which allows storing many small pieces of data, but with limitations that prevent it scaling to the point that S3 becomes economical. In this post I…
I've previously written that I don't like the CA LP's reasoning on some of tomorrow's ballot measures. I still think their arguments are only appealing to hard line libertarians, but I've been persuaded by other arguments to vote against everything tomorrow, and I encourage you to do the same. I was marginally in favor of the rainy day fund aspects of Prop 1A, but as you can see from the links in…
I'm following twitter closely tonight to see how the American Idol prediction got picked up. And what I don't understand is who are all these people who are retweeting it? They don't seem like real accounts. Or at least, not accounts that anyone who actually read twitter would be interested in following. If you search twitter for Biz360 Analysis of Social Media you see hundreds of tweets, all…
I'm writing this post as I do it. My starting point is a fresh install of Nagios 2 running on an ubuntu server in EC2. My goal is to get it monitoring some web pages. I've found poor documentation, so I'm writing this out as I go. First, you must define the command you want to use (I think). Here's my definition: define command { command_name check_http_port command_line $USER1$/check_http -h…
This is something I stumbled across while browsing the source for Hive (Hadoop ). I normally define my properties and default values using something like this: private static final String CLASSIFIER_MODEL_LOC_PROP = "models.classifier"; private static final String CLASSIFIER_MODEL_LOC_DEFAULT = "config/classifier.ser"; It works well enough, but it's a little long, and it is sort of repetition --…
The Libertarian Party of California has taken a position I think is misguided on the May 19th ballot measures . They have opposed props 1D and 1E, but these are probably a good compromise to improve California's budget situation. What 1D and 1E do is to move funding from two specially financed programs into the general fund. 1D would take tobacco tax money currently used for anti-tobacco…
My original blog was based what was available as a one click install when I set up the domain in 2006. Times have changed, and even though I don't intend to become a professional blogger, I do see value in building myself as a brand, and a well done blog is part of that. Nucleus may be great for some people, but I wanted to change hosting providers (I'm now at Dreamhost), and so I would need to at…
Here s what my linkedin profile says: I'm not looking to change jobs right now, but if you think you can change my mind, I'm open to hear from you if it's a start up with a good business plan within 15 minutes of San Mateo, you are not a recruiter, and you specifically describe the position you are trying to fill. All of you recruiters with "wonderful opportunities" with "several local companies"…
Sometimes a casual pick can lead to big problems. Early on in developing our new social media product we decided to use JSON for storing our data in HDFS. This idea has worked out really well for us. The performance is good, it's easy to read if you want to take a peak at a file by hand, and there are libraries for every language under the sun, making it easy to write streaming jobs in your…
Santa Cruz County says anyone can sign a ballot argument (from http://www.votescount.com/books/argument.pdf ) Signers of arguments for or against a county, school, or special district measure do not have to meet the criteria listed above. The filer of the argument must meet the criteria above; however, anyone may sign the argument. Shasta county agrees: Arguments and Rebuttals must be accompanied…
California s proposition 8, which says that marriage is only between a man and a woman, illustrates fundamental flaws in democracy. The right answer to the issue of gay marriage is the same as the answer to the issue of what movie I should watch tonight no one s business but mine and my wife s. I think that the vast majority of Californian s would agree with this position if asked, but social…
I m on Twitter now. I don t claim to understand the hype, yet. It reminds me of a bit in Vernor Vinge s Marooned in Realtime , one of the novels to popularize the Singularity. One of the characters describes being totally cut off from society compared to the state of the art which was augmenting your brain to be able to think with several of your colleagues at once. We aren t quite there yet, but…
This is just over the top for the most meaningless technobabble in a single sentence I ve ever read. From the January Amazon Web Services newsletter: RunMyProcess delivers a cutting edge and ergonomic "BPM as a service" platform, enabling rapid SaaS integration and workflow applications design.
It applies equally well to biology and to software development. It's easy to get everyone into a room to talk about coding standards and get everyone to agree that they are going to comment every public method, or that they are going to write good unit tests for every piece of code big enough to break. What's hard is getting people to follow through, and it's hard because it requires being an…
The bailout is in the news. We get new info daily, and it s still in flux. If you believe in small government, or even responsible government, or even some semblance of a non-corrupt government, these are the key points to make when this subject comes up in conversation. Note that I wrote this on Saturday, 20 Sept, and things may have already changed (I don t have a staff to figuring things out…
Wells Fargo wants to make it easy for you to do your banking online. They don t want you to deal with little inconveniences like having to enter your password correctly. I m sure that sometime in the past, you ve been told it s good to have both uppercase and lowercase letters in your password. I heard a rumor that Wells Fargo ignores this, tested it, and have confirmed that Wells Fargo will…
ICANN has opened a process to allow new top level domains. I just wanted to be the first one to make the obvious pun. ICANN Announcement
I use Subversion to manage documents and projects at home. I ran into a problem when I changed the IP I had assigned to the box hosting the svn repository. It isn t obvious how to change where Eclipse keeps track of this information. I found some replies to this old message , which got me to try the Switch then Relocate commands in TortoiseSVN. This made it possible to synchronize, but I wasn t…
It s almost impossible these days to find a monitor or laptop that isn t wide screen. They are even making their way into bed room size televisions. Is this the wonderful progress of technology, bringing us fantastic products for lower prices every day? Not really. It s actually just a marketing ploy to let them use a bigger number, much like the Athlon 4800+ which actually runs at 2500mhz.…
I m running for office because I m pissed off. I m pissed off at police that kick down doors using military tactics with no concern for the constitution, under the reasoning that accidentally shooting a few innocent bystanders is a small price to pay to make sure AIDS patients can t smoke marijuana. I m pissed off at having 15% of my earnings go to fund a retirement plan which will most likely go…
Looking for the perfect gift for the capitalist on your list? Try this Milton Friedman T-Shirt available from Arthur's Hall of Viking Manliness . They also have a good article How tree-Hugging hippies are destroying our environment . You gotta love any website that offers a manly answer to Oprah s Book Club .
Given the overhead for sales taxes, income taxes, etc etc, it takes about 2 hours for me to earn enough to buy one hour of the time of labor from someone who makes the same salary as me. Let s assume someone makes $20 / hr. Let s assume that he works for someone who makes something I want. It costs $20 for parts, overhead of the facilities, tools, etc. It takes 1 hour of the worker s time. So they…
I m guessing that either this place recently changed ownership, or just that the completely incompetent work the weekend shift. Mangiare was completely empty this Saturday morning, and it s no surprise since the service is the worst I have ever received. There were eight of us. The first person to receive his food was about 20 minutes after we ordered (at the register). The last person received…
When I got my ballot pamphlet a few weeks back, I was disappointed to see that they were moving to electronic voting. I think the chain of reasoning here is pretty clear: Senile idiots in Florida can't figure out how to operate paper. Electronic voting systems are even harder to use for easily befuddled retirees. Electronic voting systems cost a lot more and make more money for government…
So close one second per mile and I would have been sub-4:00. It s close enough that I m satisfied for now though. I don t plan on running another marathon for quite a while. I did my training better this time around. I was following something similar to the FIRST Training Program , but I wasn t as strict about the pace of my workouts. I did MWF bicycle, TThS run, with Tuesday being intervals (1/2…
I m currently at the Singularity Summit 2007 at San Francisco s Palace of Fine Arts. I ll post updates during the day. I will eventually (swear to the flying spaghetti monster), put these into a more coherent form. But here s some things that stuck in my head as a relative outsider to futurism per se, but someone fairly familiar with AI and longtime reader of the sl4 list. Some of these are my…
I actually won a game of chess online. I m surprised, because my general experience has been that it s all hard core chess geeks who play online, and they would slaughter someone like me who thinks that Ruy Lopez is Jennifer s brother. But after warming up for a few weeks playing my girlfriend (she s on 23 games now, about 16 of them against me, I ve lost twice due to not paying attention while in…
I received the following in email from SF Atheists, who got it from American Atheists, who received it from Kathleen Johnson, who explains who she is in the first paragraph. Thought you'd be interested in this report of the first-ever meeting of Atheist service-members in Iraq under the umbrella of the MAAF-Iraq chapter of the Military Association of Atheists and Freethinkers. This meeting was put…
I finished the SF Marathon in 4:17:46. I m very disappointed. I ran the first half in 1:50, way too fast, and just died on the second half. I haven t yet signed up for the Silicon Valley Marathon , but I am planning on running it to redeem myself and get an acceptable sub-4:00 time.
Suzuki says satori is irrational, it cannot be explained. Logical positivism says that we can only talk meaningfully about things which refer to experience. See for example Hempel s The Empiricist Criterion of Meaning in Ayer's Logical Positivism Are these incompatible? I claim they are not, because even holding to the requirements of the positivists, there is still an area that allows things…
Finished Bay to Breakers in 58:07, 773rd out of 23692. It hurt. Next year I may do it more in the spirit , meaning with a camelbak full of wine.
In this post titled Crucify Me , Patrick writes, "Why?" The question becomes predictable, almost routine when someone enters my car. They immediately seem to notice the polished steel hook hanging from my rearview mirror, and inevitably want an explanation. Each time I hear that annoying three letter word, I'm almost compelled to launch into some sort of horribly graphic and disturbing…
Juan Pablo Bernal was born in San Francisco about 1810, son of José Joaquin and Maria Josefa Sanchez (both Anza settlers), married Rafaela Feliz, granddaughter of Joaquin Ysidro Castro (Anza settler) and (supposedly) daughter of Maria Antonia Amador daughter of Pedro Amador (Portola 1769 to Monterey). José de los Reyes Berreyesa left Martínez in a boat for San Rafael in company with the two young…
Libertarians like to point to polls that suggest a lot of Americans are libertarians , based on surveys like the Advocates' WSPQ . Then they sit around wondering why no one votes for Libertarians. Surely, they say, it must be because people are not aware of what we stand for. Let s imagine a situation where everyone forgot what the traditional parties are about, and had to consider all the…