[Cs254f11] More performance issues with a particular function
Lee Spector
lspector at hampshire.edu
Thu Nov 17 13:19:57 EST 2011
I agree that it doesn't seem like that much math should slow things down that much.
Heres one thing: it looks like you're doing more work than necessary with all of that filtering and counting in score-against-scale, and that it'd be faster (not sure how much) just to look for the (normalized) note in the (normalized) transposed scale:
(defn score-against-scale
"Compares the given note against scale (which should be a list of values)
and key and returns 1 if it is in the scale, 0 if not"
[note scale key]
(some #{(mod note 12)}
(map #(mod (+ % key) 12) scale))
Incidentally, if you wrote a normalize function (just mods by 12) and a transpose function (just adds a constant to everything) then this would look neater. And I think some utilities like that would be useful elsewhere as well.
-Lee
On Nov 17, 2011, at 1:01 PM, Wm. Josiah Erikson wrote:
> Interesting. One of the problems I'm getting, though only very occasionally, is an int buffer overflow, but that's probably because I'm casting everything to an int explicitly.
>
> I'm still slightly confused about why my old way took forever - I wouldn't think a 12,000 additions and 24,000 mods would take five minutes - there must be something else I don't understand. However, my code is now pretty fast (67 times faster), so I clearly fixed SOMETHING. If you want to take a look at it with me (it's all quoted in this message) and tell me what I'm missing, I'd be curious.
>
> I might try using the latest version of clooj and see if anything is any faster - my code is now pretty fast, but faster is always better!
>
> -Josiah
>
>
> On 11/17/11 11:59 AM, Lee Spector wrote:
>> I haven't had a chance to check this in detail, but if you've determined that the delay is really in the arithmetic, of which you are presumably doing gobs, then you might like to know that improving the speed of numerics was one of the major changes in Clojure 1.3.
>>
>> You may recall that on one of the first classes I showed a Clojure factorial function, did a factorial of a large number, producing a result with hundreds of digits, and said "isn't it cool that you don't have to worry about overflow or errors from this kind of thing?" I do think it's cool, and I usually prefer that coolness, but there's a cost: under the hood lots of things have to be checked and numeric representations converted to make everything go smoothly, and this costs memory and time.
>>
>> Clojure 1.3 defaults to faster, less cool (IMHO) numerics, and provides special operators (like +', *', etc.) to get the "auto-promoting" integers of 1.2. Which I find slightly irritating BUT I concede that sometimes A) speed matters, and B) you know that huge numbers won't arise. There was a fair amount of squabbling on the clojure list about which behavior should be the default, and the winner was "fast".
>>
>> So: if you are up for it you might try working with 1.3 -- one way of doing this is just to use the latest clooj -- and see if that makes things faster (and if it breaks anything else that you're doing :-).
>>
>> -Lee
>>
>>
>> On Nov 16, 2011, at 9:03 PM, Wm. Josiah Erikson wrote:
>>
>>> mod must take a long time, because I thought about how to reduce the number of them on the way home, came up with this:
>>>
>>> (defn is_it_a_member
>>> "Takes number and a collection, returns 1 if number is in the collection, 0 if it isn't"
>>> [num coll]
>>> (count (filter #(= (mod num 12) %) coll)))
>>>
>>> (defn percent_in_mixolydian
>>> "Returns the percentage of real notes in the current song that are in the mixolydian scale,
>>> assuming that the most common note is the root"
>>> []
>>> (let [modded-scale (map #(mod (+ (most_common_real_note) %) 12) (:mixolydian scales))]
>>> (int( *( / (reduce + (map #(is_it_a_member % modded-scale) (get_real_notes (get_note_lines song))))
>>> (number_of_real_notes)) 100))))
>>>
>>> And now:
>>>
>>> (time (percent_in_mixolydian))
>>> "Elapsed time: 251.489 msecs"
>>> 51
>>>
>>> A bit more acceptable!
>>>
>>> -Josiah
>>>
>>>
>>> On 11/16/11 7:28 PM, Wm. Josiah Erikson wrote:
>>>> Hey guys,
>>>> So I'm having a very serious performance issue with a particular function that I'm writing to find out what percentage of the notes in a given song are in a given scale.
>>>>
>>>> So first, the relevant info to understand what I'm doing:
>>>>
>>>> ;Define some scales, relative
>>>> (def scales {
>>>> :mixolydian '(0 2 4 5 7 9 10)
>>>> :lydian '(0 2 4 5 7 9 11)
>>>> :major '(0 2 4 5 7 9 11)
>>>> :pentatonic '(0 3 5 7 10)
>>>> :blues '(0 3 4 5 6 7 10)
>>>> })
>>>>
>>>> (defn score-against-scale
>>>> "Compares the given note against scale (which should be a list of values) and key and returns 1 if it is in the scale, 0 if not"
>>>> [note scale key]
>>>> (count(filter #(= (mod note 12) (mod (+ %1 key) 12)) scale)))
>>>>
>>>> ;Then this takes forever:
>>>>
>>>> (defn percent_in_mixolydian
>>>> "Returns the percentage of real notes in the current song that are in the mixolydian scale,
>>>> assuming that the most common note is the root"
>>>> []
>>>> (/
>>>> (reduce + (map #(score-against-scale % (:mixolydian scales) (mod (most_common_real_note) 12))
>>>> (get_real_notes (get_note_lines song))))
>>>> (number_of_real_notes)))
>>>>
>>>>
>>>> Now, I've done a bunch of troubleshooting, and (most_common_real_note) returns quickly, as does (number_of_real_notes), and both of those functions actually also call (get_real_notes (get_note_lines song)), so that's not the issue - it's the
>>>>
>>>> (map #(score-against-scale % (:mixolydian scales) (mod (most_common_real_note) 12))
>>>> (get_real_notes (get_note_lines song)))
>>>>
>>>> that takes forever. This returns immediately, since it creates a lazy sequence:
>>>>
>>>> (def a (map #(score-against-scale % (:mixolydian scales) (mod (most_common_real_note) 12))
>>>> (get_real_notes (get_note_lines song))))
>>>>
>>>> But then if I do:
>>>>
>>>> (def b (vec a))
>>>>
>>>> To make it un-lazy, it never returns, or rather, it takes so long that I haven't let it return yet.
>>>>
>>>> (count a)
>>>>
>>>> Also takes forever, I suppose for the same reason. Some math:
>>>>
>>>> (time (score-against-scale 4 (:mixolydian scales) (mod (most_common_real_note) 12)))
>>>> "Elapsed time: 26.771 msecs"
>>>>
>>>> (number_of_real_notes)
>>>> 12166
>>>>
>>>> (/ (* 0.026 12166) 60)
>>>> 5.271933333333333
>>>>
>>>> Humph. So it will take 5.2 minutes just to run through my song and compare each note to the mixolydian scale? What am I doing wrong?
>>>>
>>>> Thanks for any suggestions,
>>>> -Josiah
>>>>
>>>>
>>>> _______________________________________________
>>>> Cs254f11 mailing list
>>>> Cs254f11 at lists.hampshire.edu
>>>> https://lists.hampshire.edu/mailman/listinfo/cs254f11
>>> _______________________________________________
>>> Cs254f11 mailing list
>>> Cs254f11 at lists.hampshire.edu
>>> https://lists.hampshire.edu/mailman/listinfo/cs254f11
>> --
>> Lee Spector, Professor of Computer Science
>> Cognitive Science, Hampshire College
>> 893 West Street, Amherst, MA 01002-3359
>> lspector at hampshire.edu, http://hampshire.edu/lspector/
>> Phone: 413-559-5352, Fax: 413-559-5438
>>
>
> --
> Wm. Josiah Erikson
> Network Engineer
> Hampshire College
> Amherst, MA 01002
> (413) 559-6091
>
--
Lee Spector, Professor of Computer Science
Cognitive Science, Hampshire College
893 West Street, Amherst, MA 01002-3359
lspector at hampshire.edu, http://hampshire.edu/lspector/
Phone: 413-559-5352, Fax: 413-559-5438
More information about the Cs254f11
mailing list