https://en.wikipedia.org/wiki/Asteroid
Asteroids, big rocks floating around our solar system. Sometimes they hit Earth and kill the dinosaurs. The name derived from 'aster' (star) 'oid' (like), because they were star-like in being points of light in the sky that didn't resolve into circles with the telescopes of the era in which they were identified, while those telescopes did show noticably circular planets and the rings of Saturn. But they're not stars, because stars don't move (on human time scales).
With varying quality we have data on 504,000 to 780,000 asteroids. We're discovering and tracking an additional 2000 near-Earth objects every year.
Telescopes and Surveys. We have a couple telescopes doing non-stop survey work of the sky in Arizona and Hawaii, there are and have been others, and the LSST in Chile, coming online around 2021, looks awesome. These surveys image the whole sky every few days and software can automatically check, "hey, did that speck of light move?". If yes, it might be an asteroid. These projects are already moving "big data" with substantial storage and computation needs, and the LSST is going to take that up a big notch.
I think it's cool that we have telescopes devoted full time to scanning the sky, and big data backing them up doing analysis. This kind of research makes a bunch of our sci-fi future possible. They have short lists of a few dozen asteroids a relatively short hop from Earth that we could mine. They have lists of asteroids that might impact Earth that we can keep an extra close watch on them so that we don't go the way of the dinosaurs. From the apparent distribution of objects in our solar system, we probably know about over 90% of the 1km+ objects that might hit earth, and better technology and techniques are pushing that coverage higher and object size threshold lower (future goals of a couple projects are all the objects over 140m or over 30m). All this for a global budget of just a few tens of millions of dollars. We can do this. When we have it together, humanity is awesome.
Tuesday, July 10, 2018
Saturday, May 26, 2018
CPU MFLOPS Test in 6 Languages
Once upon a time my dad worked on supercomputers at Los Alamos National Laboratory and sometimes he would spend a minute running a little test to see about how fast a new supercomputer was. I kept that test, flops.c, very much as I received it in 1992. Your phone is probably much faster than the fastest supercomputer of 1992.
This isn't a very good benchmark, but it's simple and easy. It runs a few basic numeric algorithms with a known number of adds, subtracts, multiplies, and divides, and figures out how many floating point operations per second your CPU can do. It doesn't take advantage of multi-core or SIMD instructions. It doesn't exercise the memory system and probably all fits in L1 cache. It just tests how fast your CPU can do math (and that your compiler isn't terrible at making that happen).
Over the years I transliterated flops.c into other languages to test them and their compilers and interpreters. Python has a pretty slow interpreter, around 1-5% of the speed of C. Javascript got amazingly good and can run 80-90% of the speed of C. Java got to that speed around 2007 or 2008. The Go compiler is surprisingly good for a newer language, it knows some tricks GCC doesn't. Julia is a newer language that should have the potential for running full speed but apparently still needs some tweaking (as of 2018-05).
Note that Go and Julia have some compiler trick the rest are missing. Test 2 is to calculate Pi from atan(1.0) by Taylor series. Without reading the assembly I'm guessing there's some fused-multiply-add going on at least. GCC 7.3.0 with -O3 didn't get it. Go reports 147 Gigaflops on test 2, and Julia 22 Gigaflops. Everything else is showing my cpu in the 5-6 Gigaflop range.
Sources on github:
https://github.com/brianolson/flops
This isn't a very good benchmark, but it's simple and easy. It runs a few basic numeric algorithms with a known number of adds, subtracts, multiplies, and divides, and figures out how many floating point operations per second your CPU can do. It doesn't take advantage of multi-core or SIMD instructions. It doesn't exercise the memory system and probably all fits in L1 cache. It just tests how fast your CPU can do math (and that your compiler isn't terrible at making that happen).
Over the years I transliterated flops.c into other languages to test them and their compilers and interpreters. Python has a pretty slow interpreter, around 1-5% of the speed of C. Javascript got amazingly good and can run 80-90% of the speed of C. Java got to that speed around 2007 or 2008. The Go compiler is surprisingly good for a newer language, it knows some tricks GCC doesn't. Julia is a newer language that should have the potential for running full speed but apparently still needs some tweaking (as of 2018-05).
Sources on github:
https://github.com/brianolson/flops
Monday, August 21, 2017
2017 eclipse Raspberry Pi pinhole box
I recorded the 2017 eclipse from where I was in Boston using a pinhole box with a Raspberry Pi 3 and 8MP camera in the box.
Pinhole at top left, display screen at bottom left, rpi, camera, and battery.
Here's the resulting video, 10 seconds per frame displayed at 30 frames per second for a 300x speedup.
It never quite focused right. If I were to try this again (in 7 years, with a rpi version 9 and whatever upgraded camera comes with it then) I'd try harder to seal off the light leaks from the box. I wasn't on my usual network and couldn't log into the rpi to get images as it went so I had to open the box now and then to reposition and track the sun.
Here's the resulting video, 10 seconds per frame displayed at 30 frames per second for a 300x speedup.
It never quite focused right. If I were to try this again (in 7 years, with a rpi version 9 and whatever upgraded camera comes with it then) I'd try harder to seal off the light leaks from the box. I wasn't on my usual network and couldn't log into the rpi to get images as it went so I had to open the box now and then to reposition and track the sun.
Wednesday, October 12, 2016
Postgres global locking problem
It seems that PostgreSQL can't `VACUUM FULL` one database and `pg_dump` another database at the same time. These things were going on in a database I was observing and I saw things back up waiting for an exclusive lock on pg_catalog.pg_db_role_setting. I found in the documentation that `VACUUM FULL` is one of the only things that takes out such an exclusive lock such that nothing else can even do a read-only `SELECT` off the table that's locked. The log file noted "waiting for AccessShareLock on relation ..." and a oid number which turned out to be pg_catalog.pg_db_role_setting.
So, that's annoying, and a reason that we're going to have to continue splitting up postgres servers into more single-function servers, because there isn't actually enough internal isolation sometimes.
Monday, April 25, 2016
Democracy Still Needs Fixing
Hundreds of thousands of people wanted to vote in the primaries this year, but were denied by the parties, and we let this happen. We let private clubs determine if someone can vote. Isn’t that absurd?
I’d kinda like to get rid of publicly funded primaries. Why do we spend millions of dollars of public money to run a private club’s organizational internal vote?
Maybe we make a deal, in exchange for public funding of the primary vote, the party has to let anyone vote in their primary (regardless of registration in that or any other party). If the X party want a closed primary, they can run it (and pay for it) themselves. Maybe the Internet Party will have an online primary, that’d be cool, eh?
Furthermore, people should be insulted that they didn’t get to vote in the primary because a Party didn’t want their vote, didn’t want their participation. The parties should be worried that they’re insulting and driving away people. Except that actually they both seem happy to play the game of trying to disenfranchise as many voters as possible (who might possibly vote for the other side).
I want the answer to be more democracy, not less. We could break the lock held by The Two Parties if we changed ballot access laws and used a rankings or ratings ballot in the general election. A ‘primary’ would be a private function of a party that gets a slot on the ballot because it has some number of registered members. Anyone else can get on the ballot by signature petition. On the ballot you get to rank your choices 1st, 2nd, 3rd, etc…; or rate them on a scale of 0..10. Our current focus on primaries is an artifact of needing to get down to two choices on the final ballot, but that’s not necessary with a better ballot where you can express yourself on the whole field (or as much of it as you care to).
Break the two party duopoly. Another rant, because it’s Monday.
I’d kinda like to get rid of publicly funded primaries. Why do we spend millions of dollars of public money to run a private club’s organizational internal vote?
Maybe we make a deal, in exchange for public funding of the primary vote, the party has to let anyone vote in their primary (regardless of registration in that or any other party). If the X party want a closed primary, they can run it (and pay for it) themselves. Maybe the Internet Party will have an online primary, that’d be cool, eh?
Furthermore, people should be insulted that they didn’t get to vote in the primary because a Party didn’t want their vote, didn’t want their participation. The parties should be worried that they’re insulting and driving away people. Except that actually they both seem happy to play the game of trying to disenfranchise as many voters as possible (who might possibly vote for the other side).
I want the answer to be more democracy, not less. We could break the lock held by The Two Parties if we changed ballot access laws and used a rankings or ratings ballot in the general election. A ‘primary’ would be a private function of a party that gets a slot on the ballot because it has some number of registered members. Anyone else can get on the ballot by signature petition. On the ballot you get to rank your choices 1st, 2nd, 3rd, etc…; or rate them on a scale of 0..10. Our current focus on primaries is an artifact of needing to get down to two choices on the final ballot, but that’s not necessary with a better ballot where you can express yourself on the whole field (or as much of it as you care to).
Break the two party duopoly. Another rant, because it’s Monday.
Saturday, February 13, 2016
Ruby Scope
Ruby has perhaps a new variation on the bad old LISP thing of some functions evaluating in the context where they're called rather than in a self-contained scope.
class Foo
def initialize(a)
@a = a
end
def do_wat(m)
wat_helper(m)
end
end
def wat_helper(n)
@n = n
@a = "lol"
end
f = Foo.new(1)
p f #
f.do_wat(2)
p f #
wat_helper() isn't a member function of any class, so where do its member variables @n and @a bind? Whatever instance context it's called from, apparently.
This is powerful and terrible. On the one hand I could write some nice utility functions for maintaining a tree or a doubly-linked-list and inside any instance maintain @prev, @next, @less, @greater, @parent, and so on. That's great for re-use of the data structure utility functions.
But it's also terrible because I now kinda need to not trust any function I call because it might clobber or pollute my instance variables. There's a notion out there that one should code-review any open source project you're thinking about importing to see if its code is up to your standards. Practically things tend to get in on reputation and basic demonstration of working right. But if code I'm importing can potentially clobber my data structures in this weird way I feel like I should be extra suspicious of it.
https://bolson.org/~bolson/2016/wat_scope.rb
class Foo
def initialize(a)
@a = a
end
def do_wat(m)
wat_helper(m)
end
end
def wat_helper(n)
@n = n
@a = "lol"
end
f = Foo.new(1)
p f #
f.do_wat(2)
p f #
wat_helper() isn't a member function of any class, so where do its member variables @n and @a bind? Whatever instance context it's called from, apparently.
This is powerful and terrible. On the one hand I could write some nice utility functions for maintaining a tree or a doubly-linked-list and inside any instance maintain @prev, @next, @less, @greater, @parent, and so on. That's great for re-use of the data structure utility functions.
But it's also terrible because I now kinda need to not trust any function I call because it might clobber or pollute my instance variables. There's a notion out there that one should code-review any open source project you're thinking about importing to see if its code is up to your standards. Practically things tend to get in on reputation and basic demonstration of working right. But if code I'm importing can potentially clobber my data structures in this weird way I feel like I should be extra suspicious of it.
https://bolson.org/~bolson/2016/wat_scope.rb
Friday, February 5, 2016
More Rails WAT
If something appears to be in English plural form, Rails will treat it differently. I have more or less come to peace with the Go convention of Capitalized Fields And Methods being 'public' and lower case fields and methods being 'package' scope (in Java/C++ terms). It's weird, but okay I guess. It's a hard and enforced rule, which is probably better than the Python convention of methods and fields starting with '_' being visible but please don't use them please treat them as package or private and not for you.
In my config/routes.rb I wound up with two lines via various experiments in building a rails app:
resource :event
resources :event_wats
I wrote a migration by hand which does `create_table :event ...` and I called the setup tool `rails generate scaffold EventWat ...` to create a second table with a migration `create_table :event_wats ...`. I read about this utility in a new tutorial I was reading today and wondered what all I would get from that automated setup. It writes a lot of files!
I also discovered that 'resource' vs 'resources' is a thing. A 'resource' is apparently some kind of singleton. I guess there would only ever be one row of it in the database table. 'resources' get the rest of the REST path bits to deal with multiple rows in the table, list an index of things, etc.
The "Convention over Configuration" doctrine necesitates tools like this! It's like the canonical form of certain special C++ functions, only times twenty. A table of a certain name needs route, model, controller, and view files and classes of certain names.
Rails pedagogy seems to be heavy on tutorial and light on reference documentation. I tried to build my 'event' table based on reference documentation and I missed that the tools had moved on and using `rails generate scaffold` was the new right way to set up things. The reference documentation for setting up a table missed talking about how it ties into the rest of the documentation, and whether the table name should be plural or not and where plural naming matters.
"Convention over Configuration" makes me mad. If I `rails generate scaffold` I get a full set of files that hang together by convention. If I piece things together manually I’ll probably guess the convention wrong at some point and I can’t configure things to hang together because there is no configuration. I just need to rename my classes or tables or files or something until I meet the strange demands of the evil wizard who designed this ‘convention’.
CoC makes the learning curve steeper. It's like I can't just learn one part of the system, I have to learn the whole system at once and the convention it lives by. CoC hides what is going on, and that is not good. I want to have all the moving parts visible and traceable by inspection of the code in a project. I don't want spooky action at a distance. I don't want things to work magically. If something is going to be made easier, I want the APIs designed to make explicit configuration as easy as possible.
In my config/routes.rb I wound up with two lines via various experiments in building a rails app:
resource :event
resources :event_wats
I wrote a migration by hand which does `create_table :event ...` and I called the setup tool `rails generate scaffold EventWat ...` to create a second table with a migration `create_table :event_wats ...`. I read about this utility in a new tutorial I was reading today and wondered what all I would get from that automated setup. It writes a lot of files!
I also discovered that 'resource' vs 'resources' is a thing. A 'resource' is apparently some kind of singleton. I guess there would only ever be one row of it in the database table. 'resources' get the rest of the REST path bits to deal with multiple rows in the table, list an index of things, etc.
The "Convention over Configuration" doctrine necesitates tools like this! It's like the canonical form of certain special C++ functions, only times twenty. A table of a certain name needs route, model, controller, and view files and classes of certain names.
Rails pedagogy seems to be heavy on tutorial and light on reference documentation. I tried to build my 'event' table based on reference documentation and I missed that the tools had moved on and using `rails generate scaffold` was the new right way to set up things. The reference documentation for setting up a table missed talking about how it ties into the rest of the documentation, and whether the table name should be plural or not and where plural naming matters.
"Convention over Configuration" makes me mad. If I `rails generate scaffold` I get a full set of files that hang together by convention. If I piece things together manually I’ll probably guess the convention wrong at some point and I can’t configure things to hang together because there is no configuration. I just need to rename my classes or tables or files or something until I meet the strange demands of the evil wizard who designed this ‘convention’.
CoC makes the learning curve steeper. It's like I can't just learn one part of the system, I have to learn the whole system at once and the convention it lives by. CoC hides what is going on, and that is not good. I want to have all the moving parts visible and traceable by inspection of the code in a project. I don't want spooky action at a distance. I don't want things to work magically. If something is going to be made easier, I want the APIs designed to make explicit configuration as easy as possible.
Subscribe to:
Posts (Atom)