tag:blogger.com,1999:blog-48651153277430736662023-11-15T18:25:36.955+01:00DeveradoSome stuff you might find useful -- if you found this in your favorite search engine ; )Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-4865115327743073666.post-89398797847207730242013-11-07T11:29:00.001+01:002013-11-07T11:29:34.540+01:00Evil Cassandra Unavailable Exception and a few SolutionsThis is a nasty situation when learning cassandra: You have a <a href="http://cassandra.apache.org/">cassandra</a> cluster up, you try running your clients (server or command line) and nothing seems to work (example from cassandra-cli):<br />
<br /><blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">[default@blub] list user;<br />Using default limit of 100<br />Using default column limit of 100<br />null<br />UnavailableException()<br /> at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassa....</span></blockquote>
<div>
<br />Basically the reason is that your query's <a href="http://www.datastax.com/docs/1.1/dml/data_consistency">consistency level</a> cannot be satisfied by the cassandra cluster. Couple of fixes, depending on your situation:<br />
<br />
<ol>
<li>If your configured replication factor is 1 or your cluster has too few nodes: Use a lower consistency level. Test in cassandra-cli:<br /><span style="font-family: Courier New, Courier, monospace;">[default@blub] consistencylevel as one;<br />Consistency level is set to 'ONE'.</span><br /></li>
<li>If you started with a single node and recently added another, or you had to replace a node, or you changed the replication factor: Run repair on the node(s):<br /><span style="font-family: Courier New, Courier, monospace;">apache-cassandra-1.2.5$ ./bin/nodetool -h localhost repair -l<br />[2013-11-07 11:09:04,369] Starting repair command #1, repairing 1 ranges for keyspace blub<br />[2013-11-07 11:09:04,376] Repair command #1 finished<br />[2013-11-07 11:09:04,387] Nothing to repair for keyspace 'system'<br />[2013-11-07 11:09:04,393] Starting repair command #2, repairing 1 ranges for keyspace system_auth<br />[2013-11-07 11:09:04,394] Repair command #2 finished<br />[2013-11-07 11:09:04,402] Nothing to repair for keyspace 'system_traces'</span><br /></li>
<li>If you run with<br /><span style="font-family: Courier New, Courier, monospace;">org.apache.cassandra.locator.NetworkTopologyStrategy</span> and multiple data centers: Careful with consistency level QUORUM: In my experience it requires quorums in all data centers! So if you have a data center with replication set to 1 you will always get UnavailableException:<br />
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">CREATE KEYSPACE <span class="s1">vcodeks </span> with placement_strategy = 'org.apache.cassandra.locator.NetworkTopologyStrategy'</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> strategy_options = {dc1:2, dc2:1};</span><br /><br />In this case it is better to use consistency level TWO. If you only care about one data center you might be happy with LOCAL_QUORUM if you configure all your client's DCs. But be aware that in the upper example LOCAL_QUORUM clients with dc set to <span style="font-family: Courier New, Courier, monospace;">dc1</span> won't be available if only one node in <span style="font-family: Courier New, Courier, monospace;">dc1</span> failed, even though you have a combined 3 replicas over all. </div>
</li>
</ol>
</div>
Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.com0tag:blogger.com,1999:blog-4865115327743073666.post-69985690916187445742012-06-22T17:45:00.000+02:002012-06-22T17:46:20.501+02:00Moving to deverado.wordpress.comMoving to <a href="http://deverado.wordpress.com/">deverado.wordpress.com</a> because commenting was difficult.<br />
<br />
I have third party cookies disabled, and I like it that way. Blogger doesn't work like that, just tested it. Here is <a href="http://productforums.google.com/forum/#!topic/blogger/iwWeWeBj-qs/discussion">the discussion that tipped me of</a>.<br />
<br />
I like discussions and disabling third party cookies -> move decision clear. Always wanted to try tumblr and Wordpress.<br />
Tumblr: Out because requires people to have accounts to comment on posts<br />
Wordpress: Powerful - but I'll miss the tree-archive widget!Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.com0tag:blogger.com,1999:blog-4865115327743073666.post-19529177338661560562012-06-21T09:17:00.001+02:002012-06-21T09:17:04.980+02:00Chef's awesome - but bootstrapping can be a painBootstrapping chef nodes in a bar bones virtual server environment took me some time to figure out. I had especially issues with the hostname not being recognized.<br />
<br />
Problem is that in my bare-bones vservers the servers come without a useful hostname/domainname setting.<br />
<br />
Finally now everything works. I did:<br />
<span style="background-color: white;"><br /></span><br />
<br />
<ol>
<li><span style="background-color: white;">Create the vserver(s)</span></li>
<li><span style="background-color: white;">Set up the vserver's DNS entries</span></li>
<li><span style="background-color: white;">Copy my ssh id to the server(s):<br />scp ~/.ssh/id_dsa.pub root@1.1.1.1:/root/.ssh/authorized_keys</span></li>
<li>Create my base role (check out the quickstart)</li>
<li>Add the hostname and resolv recipes of johntdyer https://github.com/johntdyer/hostname-chef https://github.com/johntdyer/resolv-chef (they have to be in you local repo as cookbooks/resolve and .../hostname). Upload them and add them to the base role's run-list</li>
<li>DNS should be ready now, so:<br />knife bootstrap hostname.domain.de -E dev -N hostname.domain.de -r 'role[base]'<br />Decide if you need the environment setting (-E) - I find it damn useful.</li>
<li>NOW BEWARE, your nodes can now be found with<br /><span style="background-color: white;">knife search "name:*" BUT<br />knife ssh "name:*" DOES NOT WORK. Maybe a bug: http://serverfault.com/questions/346418/knife-ssh-doesnt-find-my-nodes/346542#346542<br />The nodename init needs a chef-client run to propagate. So do:<br />knife ssh "name:**filter**" chef-client -E dev -a ipaddress -x root<br />Here pay attention to the <b>ipaddress</b> part. I would love to know how I should have discovered that the thing that is referred to as IP in knife search and other places is called ipaddress here...</span></li>
<li><span style="background-color: white;">Done. Check your nodes: </span>knife search node "*:*"<br />The names should be corrected now.</li>
</ol>
<br />
If anybody knows the siblings to ipaddress and where that is documented please let me know. There's a question, too: http://serverfault.com/questions/400836/what-are-the-values-for-attributes-in-knife-ssh-a-ipaddress-etc<br />
<br />Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.com0tag:blogger.com,1999:blog-4865115327743073666.post-43245594905302796592012-04-22T18:25:00.000+02:002012-04-22T18:26:25.482+02:00BSON objectid customization for sharding<br />
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Following the recommendations in <a href="http://www.mongodb.org/display/DOCS/Choosing+a+Shard+Key" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #4183c4; font: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-decoration: none;">http://www.mongodb.org/display/DOCS/Choosing+a+Shard+Key</a> I implemented these two id generators that follow the ObjectId model. I hope some day they can get driver support so that saving them in binary format will be possible.</div>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Code on <a href="https://github.com/georgkoester/shardableobjectids">https://github.com/georgkoester/shardableobjectids</a> , downloads <a href="https://github.com/georgkoester/shardableobjectids/downloads">https://github.com/georgkoester/shardableobjectids/downloads</a></div>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
ShardableObjectId: Creates a nice distribution of keys over all buckets/shards for linear write/read scaling.</div>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
ShardableObjectIdWithMoPrefix: Creates keys prefixed with yyyymm so eg. 201203 so that inserts affect only part of the index.</div>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Currently I propose just using them as string generators with the toString or toStringBase64URLSafe methods. The generated strings are safe for copy'n paste and work in most frameworks as entity ids that can be passed in the URL.</div>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Needs:</div>
<ul style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 30px; padding-right: 0px; padding-top: 0px;">
<li style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-size: 15px; font: inherit; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">BSON mongodb driver</li>
<li style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-size: 15px; font: inherit; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">apache commons codec</li>
</ul>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Limitations:</div>
<ul style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 30px; padding-right: 0px; padding-top: 0px;">
<li style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-size: 15px; font: inherit; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">design has a log of repeated code everywhere (shardable* and BSON ObjectId very similar).</li>
<li style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-size: 15px; font: inherit; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">string generation not optimized. Too lazy to change apache commons codec now.</li>
<li style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-size: 15px; font: inherit; margin-bottom: 15px; margin-left: 0px; margin-right: 0px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">depends on commons codec</li>
</ul>
<div style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font: inherit; line-height: 18px; margin-top: 15px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">
Those limitations are fine for me of course.<br />
<br />
What do you think?</div>Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.com0tag:blogger.com,1999:blog-4865115327743073666.post-39044345179425566742012-04-10T23:40:00.000+02:002012-04-10T23:40:02.722+02:00Using dust.js in play framework 2.0I'm really into trying frameworks and <a href="http://www.playframework.org/">play 2.0</a> is a really nice package. Something's always missing, and for me and play 2.0 it's <a href="http://akdubya.github.com/dustjs/">dust.js</a> templates.<br />
<br />
There's a <a href="https://github.com/typesafehub/play-plugins/tree/master/dust">half official typesafe dust sbt plugin</a>, but<br />
<br />
<ol>
<li>I cannot make it download via ivy/mave/sbt magic</li>
<li>I cannot compile it - how do I let it find play?</li>
</ol>
<div>
So had to make a shameless copy of the nice coffeescript plugin that comes with play 2.0. You can clone my <a href="https://github.com/georgkoester/Play20/compare/dust_play_sbt_plugin">dust sbt play plugin on github</a>.</div>
<div>
<br /></div>
<div>
Just call </div>
<div>
<ol>
<li>git clone git://github.com/georgkoester/Play20.git </li>
<li>cd Play20</li>
<li>git checkout dust_play_sbt_plugin </li>
<li>cd framework</li>
<li>./build</li>
<li>build-repository</li>
<li>compile</li>
<li>publish-local</li>
</ol>
<div>
and you got a version of play 2.0 with my patch. Of course you could just pull the patch too.</div>
</div>
<div>
<br /></div>
<div>
Happy playin!</div>Georghttp://www.blogger.com/profile/15419137708465665442noreply@blogger.com6