[Cs254f11] Variable scope issue
Lee Spector
lspector at hampshire.edu
Thu Nov 17 18:05:59 EST 2011
Ah, right! You're evolving the code that will call the functions, and you don't want arguments to actually appear in those evolved calls
I think that atoms are indeed one reasonable approach here. I'm not sure if your re-defing is actually making copies that would need to be garbage collected anyway -- it depends on what you're doing to produce the values that you're passing to def. But it's more than just a style thing. Def isn't designed to be used repeatedly in the way that you are, it may involve overheads that you don't need, it will require explicit "dynamic" declarations in Clojure 1.3+, and it will fail utterly in the context of multithreading.
Speaking of multithreading, the atom solution is "hygenic" and better than the re-defing approach but it still wouldn't allow multiple instances of your process to proceed concurrently on multiple cores since they'd be resetting the atom out from under each other (but at least doing so safely...). It's not immediately obvious how best to support concurrency in this situation... but my first thought is that it might involve the "binding" macro.
Second thought: Here's another, somewhat radically different approach, that doesn't involve atoms OR binding or anything else very fancy:
1. In your evolved program, before evaluating it, replace all of the calls to your "terminals" (like most_common_real_note) that must internally refer to an argument (like song) with a call that takes an argument -- that is, replace (most_common_real_note) with (most_common_real_note song).
2. Wrap this up in a function that takes song as an argument, and call that function on the current song, as I do in the error function in https://gist.github.com/1335696
-Lee
On Nov 17, 2011, at 1:46 PM, Wm. Josiah Erikson wrote:
> Well, the reason I'm using globally def'd vars is because the functions in question that rely on term-map being set are actually terminals themselves, and therefore 0-arity.
>
> I could use atoms, sure - would be it make things more efficient, because I wouldn't be creating new copies of variables all the time, or is it just a style thing because the garbage collector is pretty good?
>
> Thanks,
>
> -Josiah
>
>
>
> On 11/17/11 1:35 PM, Lee Spector wrote:
>> I'll need to see all of the code again together to give a specific answer about term-map, but in this situation it often helps to 1) consider passing things as arguments rather than thinking that they have to be the values of global vars, and 2) if it really makes sense to have them in global vars then use atoms (that you set with reset! and refer to with @ or deref -- see the section of clojinc with atoms).
>>
>> -Lee
>>
>>
>> On Nov 17, 2011, at 1:29 PM, Wm. Josiah Erikson wrote:
>>
>>> OK, that works. I find the fact that a called function doesn't inherit the calling functions variables somewhat intuitive... I mean, actually, it would probably cause problems with variable scope if it did. The reason, clearly, that this didn't work for me, is that I was passing a variable NAME, not data, as you say. I didn't quite understand that at first, but it makes perfect sense.
>>>
>>> So for anybody who is curious, create_function_map now looks like this:
>>>
>>> (defn create_function_map
>>> "Returns a map where the keys are the functions in function-list
>>> and the values are what you get when you eval said function with song set to song"
>>> [song function-list]
>>> (zipmap function-list (map #((eval %) %2) function-list (repeat song))))
>>>
>>> and my list of analysis functions (which gets passed as function-list to create_function_map) now looks like this:
>>>
>>> (def list-of-analysis-functions '(std_dev_mostcom high_note low_note average_note average_real_note
>>> most_common_real_note least_common_real_note number_of_different_real_notes
>>> average_velocity percent_in_mixolydian percent_in_lydian percent_in_pentatonic
>>> percent_in_blues percentage_of_percussion percentage_of_pitchbend))
>>>
>>> And everything works perfectly, without ever having to define song globally. Thank goodness. However, I still depend on term-map being defined globally and being redefined inside functions all the time, and would love your thoughts on how to get around that (though it works perfectly).
>>>
>>> Thanks for the explanation, Lee.
>>>
>>> -Josiah
>>>
>>>
>>>
>>> On 11/17/11 12:54 PM, Lee Spector wrote:
>>>> You are right that you shouldn't have to def things inside of defs, and that there are cleaner, better ways to do what you want (see below). (Incidentally in 1.3 you'll be forced to declare vars as dynamic if you want to re-def them.)
>>>>
>>>> The reason that let doesn't work is indeed variable scope, as your message title implies. Let is lexically scoped, meaning that only code that appears within the textual body of the let form will be able to see the binding created by the let.
>>>>
>>>> But things can get pretty confusing when you're using eval and anything for which scope matters. The fact that (def song song) works for you is surprising and must mean that the two instances of "song" in there have different references, which must have something to do with the the implementation of def, which is a "special form".
>>>>
>>>> Much better than worrying about this sort of thing is to avoid the issue by eval-ing only the function names, to get the functions themselves, and passing the rest as evaluated data. For example:
>>>>
>>>> (defn foo [x] x)
>>>>
>>>> (defn bar [x] (inc x))
>>>>
>>>> (defn baz [x] (* 2 x))
>>>>
>>>> (def fns '(foo bar baz))
>>>>
>>>> (map #((eval %1) %2) fns (repeat 7))
>>>>
>>>> => (7 8 14)
>>>>
>>>> So in create_function_map you'd do something similar, not mapping eval down a list of expressions that include variable names, but rather evaluating just the function names, and using calling each of them on the input. Another way to do that:
>>>>
>>>> (let [actual-fns (map eval fns)]
>>>> (map apply actual-fns (repeat [7])))
>>>>
>>>> => (7 8 14)
>>>>
>>>> Here I evaluated all of the function names first and put the actual function objects (not the symbols that name them) in actual-fns. Then I mapped apply down the lists of functions and repeated copies of the full argument list for each application.
>>>>
>>>> -Lee
>>>>
>>>>
>>>>
>>>>
>>>> On Nov 17, 2011, at 11:47 AM, Wm. Josiah Erikson wrote:
>>>>
>>>>> So I'm rewriting my functions that create my globally-accessible map of functions to values per song. I'm rewriting them because I figured it was bad form to make them depend on "song" being set globally - rather I could pass the value of song to them, and that would be better style. That's all fine and dandy, except that the function I've written to create my global map doesn't seem to pass the value of song that it gets passed into it on to the functions that it calls:
>>>>>
>>>>> (defn create_function_map
>>>>> "Returns a map where the keys are the functions in function-list
>>>>> and the values are what you get when you eval said function with song set to song"
>>>>> [song function-list]
>>>>> (zipmap function-list (map eval (map #(list % 'song) function-list))))
>>>>>
>>>>> (defn create_terminal_map_vector
>>>>> "This takes a list of function names and a list of parsed songs, and returns a vector of maps.
>>>>> The maps have keys that are the function names and the values are the values that said functions
>>>>> return when song is set to the corresponding song in parsed-song-list"
>>>>> [function-list parsed-song-list]
>>>>> (vec(map create_function_map parsed-song-list (repeat function-list))))
>>>>>
>>>>> (def terminal-map (create_terminal_map_vector list-of-analysis-functions parsed_songs))
>>>>>
>>>>> That last def, which is what creates the data that everything else refers to, barfs because create_function_map (yes, I know, I could roll it into create_terminal_map_vector, and I probably will eventually, but it's easier to read this way for now) doesn't pass its value of song on to the "eval" call. If I just modify create_function_map so that it does a "def song song" before the zipmap, everything is fixed. I also tried a local "let", but that doesn't work either. Why? Clearly I'm missing something obvious. I shouldn't have to def things inside functions, right?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> --
>>>>> Wm. Josiah Erikson
>>>>> Network Engineer
>>>>> Hampshire College
>>>>> Amherst, MA 01002
>>>>> (413) 559-6091
>>>>>
>>>>> _______________________________________________
>>>>> Cs254f11 mailing list
>>>>> Cs254f11 at lists.hampshire.edu
>>>>> https://lists.hampshire.edu/mailman/listinfo/cs254f11
>>>> --
>>>> Lee Spector, Professor of Computer Science
>>>> Cognitive Science, Hampshire College
>>>> 893 West Street, Amherst, MA 01002-3359
>>>> lspector at hampshire.edu, http://hampshire.edu/lspector/
>>>> Phone: 413-559-5352, Fax: 413-559-5438
>>>>
>>> --
>>> Wm. Josiah Erikson
>>> Network Engineer
>>> Hampshire College
>>> Amherst, MA 01002
>>> (413) 559-6091
>>>
>> --
>> Lee Spector, Professor of Computer Science
>> Cognitive Science, Hampshire College
>> 893 West Street, Amherst, MA 01002-3359
>> lspector at hampshire.edu, http://hampshire.edu/lspector/
>> Phone: 413-559-5352, Fax: 413-559-5438
>>
>
> --
> Wm. Josiah Erikson
> Network Engineer
> Hampshire College
> Amherst, MA 01002
> (413) 559-6091
>
--
Lee Spector, Professor of Computer Science
Cognitive Science, Hampshire College
893 West Street, Amherst, MA 01002-3359
lspector at hampshire.edu, http://hampshire.edu/lspector/
Phone: 413-559-5352, Fax: 413-559-5438
More information about the Cs254f11
mailing list