[Cs254f11] Macro confusion
Omri Bernstein
ob08 at hampshire.edu
Wed Oct 12 10:48:01 EDT 2011
On Wed, Oct 12, 2011 at 8:03 AM, Lee Spector <lspector at hampshire.edu> wrote:
>
> On Oct 11, 2011, at 10:45 PM, Omri Bernstein wrote:
>
> > Thanks Lee! Writing macros does feel quite like an art, and I seem to
> figure it out more with trial and error than with formal logic.
> >
> > Yeah, I already had a look at clojure.contrib.combinatorics. It has a lot
> of cool stuff, but not anything that completely did what I was trying to do,
> so I figured I'd make my own. And, as you said, it's a good learning
> experience.
> >
> > I did a macro so that I could pass the "for" a theoretically limitless
> vector. I could just changed the "defmacro" to a "defn", but then I would
> just have to evaluate what it returns. If I could come up with an
> alternative for using "for" as the core function, then I could probably do
> it as a function. Actually, I was originally trying to use "map", but I
> wasn't successful in getting past a combins function that could handle any
> more than two collections:
> >
> > (defn combins
> > [f c1 c2]
> > (map (fn [n] (map (partial f n) c2)) c1)
> >
> > I did a mental back-flip just getting this far, and I'm struggling with
> how to even add a third collection to the series, let alone generalize it to
> [f & colls]. Any thoughts?
>
>
> Ah -- now I see the temptation of writing a macro here. You wanted to
> construct a "for" structure and then evaluate it. Ironically, the reason
> that this would seem to call for a macro is that "for" itself is a macro,
> and this can cause the kinds of complications to which I was alluding when I
> warned about macro over-use. You can't just dynamically create the bindings
> part of a "for" and then apply "for" to it, because "for" is a macro... Of
> course "for" *is* very handy, and it has to be a macro to do what it does,
> so I'm not saying it's bad. But it can't be used as flexibly as a function
> can be used.
>
> In any event, there are a variety of ways in which you might do what you
> want without writing a macro, and the clearest (to me, right now anyway)
> would do it without using "for" at all. Here's one way that's particularly
> simple because it leverages clojure.contrib.combinatorics:
>
> (use 'clojure.contrib.combinatorics)
>
> (defn combins
> [f & colls]
> (map #(apply f %) (apply cartesian-product colls)))
>
>
Ah yes, I was using this exact same thing a while ago. I decided to come up
with something else because this seemed like a very computationally
intensive way of doing it. I don't know if that's really the case, but
mapping an apply just seems like it would be difficult. Perhaps I should
stop trying to "empathize" with the machine and actually find out whether
that's the case.
Actually, late last night, I managed to overcome my mental block and produce
a combins function. It works by building up combinations of arguments into a
massive list of partial functions, which is then called on the elements of
the last collection in the series:
(defn combins
[f & colls]
(letfn [(partial-fs+args
[f-coll arg-coll]
(flatten
(map (fn [_f]
(map (fn [_arg]
(partial _f _arg))
arg-coll))
f-coll)))]
(mapcat (fn [elem-of-last-coll]
(for
[part-f (reduce partial-fs+args [f] (drop-last colls))]
(part-f elem-of-last-coll)))
(last colls))))
It works great, from what I can tell.
>
> > As for the errors I got, I see why my fix solved the "eager evaluation"
> problem that was happening. As you said, it was trying to call \t as a
> function. What I don't understand is what the problem was when I originally
> tried casting the argument as a vector--which should evaluate to itself and
> not call \t as a function--which it didn't. However, it doesn't return the
> correct answer:
> >
> > (combins try-args [+ -] (vec (keys arg-types))
> > >> (nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
> >
> > When I did that fix, it didn't "error", but it did give me an incorrect
> result. I don't understand why it was giving me incorrect results, and I
> don't understand why doing `~colls instead of colls changed that.
>
>
> It's hard for me to reconstruct what definitions you had in place when you
> got this output. When I try I get an error because you have "test-types" in
> your "try-args" function where I think you meant to have "arg-types"....
> Maybe you had something called test-types previously? In any event, when I
> change this then from your call I get a different (maybe correct?) result:
>
> (nil
> nil
> nil
> (#<core$_PLUS_ clojure.core$_PLUS_ at 27af9da2> :number)
> nil
> nil
> nil
> nil
> nil
> nil
> (#<core$_ clojure.core$_ at 1506291d> :number)
> nil
> nil
> nil)
>
> -Lee
>
>
Weird, I went back and redid it (like you did) with the combins macro. It
does seem to work fine, which makes my previous problem vanish rather shyly.
-Omri
>
>
>
>
> >
> > -Omri
> >
> >
> > On Tue, Oct 11, 2011 at 4:50 PM, Lee Spector <lspector at hampshire.edu>
> wrote:
> >
> > One other thing that may be of interest: clojure.contrib.combinatorics
> >
> > -Lee
> >
> >
> > On Oct 11, 2011, at 4:22 PM, Lee Spector wrote:
> >
> > >
> > > Lots of good stuff in there!
> > >
> > > A few comments:
> > >
> > > - Is there really any reason to make this a macro? Why not just a
> function that takes the same "& colls" argument? As far as I can tell that
> would work just as well in this case and be less confusing. On the other
> hand, you got some good practice writing macros :-). But on the third hand
> you should only use macros when you really need them -- which is usually
> when you want to control whether code passed in as arguments gets evaluated
> (or how many times it gets evaluated). One of the downsides of writing
> macros when functions could do the job is that you can pass a function to
> another function, map it down a sequence, etc., but you can't do any such
> thing with a macro.
> > >
> > > - You may well be able to get the argument/type information that you're
> computing here from Java "reflection" rather than trying things and catching
> exceptions. I'm not sure, and again what you're doing is a cool exercise in
> any event, but you might want to look into this.
> > >
> > > - Writing macros is an art, and a pretty deep one. Paul Graham's book
> "On Lisp" is largely about the art of macro writing, and I think it's
> extremely cool. But it uses Common Lisp rather than Clojure, so some but not
> all of it will apply to Clojure.
> > >
> > > - In my macro-writing past, which was mostly in Common Lisp, I
> sometimes ended up doing the same kind of trick that you did with
> backquote-tilde, although it would have been backquote-comma in common lisp.
> Generally this happens when you want the macro's argument to be evaluated,
> but then you want that value to be quoted in the macro's expansion code. In
> your case evaluating the argument to the macro call will give (\t [1 2 3 4]
> + :42 #"ab" #{1 2 3 4} "Text is in here.") and you want that to be quoted in
> the macro expansion's call to interleave. If it isn't quoted there then when
> the expansion is evaluated it will try to call \t as a function, which gives
> the error that you saw.
> > >
> > > I think that that last point answers your question, but I also think
> that you should look into the other points, particularly the first.
> > >
> > > -Lee
> > >
> > >
> > > On Oct 11, 2011, at 3:50 PM, Omri Bernstein wrote:
> > >
> > >> As a presage to this email, I spend a lot of time explaining, and my
> question doesn't really come until the very end. Also, by the time I got to
> the end of writing , I managed to accidentally solve my original question,
> and now I'm curious whether anybody can tell me why what I did solved the
> problem.
> > >>
> > >> I've been trying to work on getting the arities for all the clojure
> functions, and what types of arguments each arity for each function will
> accept. I'm on the second part now, and I plan on going about it by trying
> all the possible combinations of argument types, and returning the types of
> the ones that don't error. So I decided to make a general-purpose macro for
> returning a list of all the possible combinations of applying a function f
> to a series of collections--using each value from the first collection as
> the first argument to f, and each from the second collection as the second
> argument to f, etc. If you understand how the "for" loop comprehension works
> (if not, maybe the examples here would help), this is an example of how the
> concept would work, using + as the function, [1 2 3] as the first
> collection, and [10 20] as the second collection:
> > >>
> > >> (for [elem1 [1 2 3] elem2 [10 20]]
> > >> (+ elem1 elem2))
> > >>>> (11 21 12 22 13 23)
> > >>
> > >> This could be generalized to:
> > >>
> > >> (defn combins
> > >> [f c1 c2]
> > >> (for [elem1 c1 elem2 c2]
> > >> (f elem1 elem2))
> > >>
> > >> (combins + [1 2 3] [10 20])
> > >>>> (11 21 12 22 13 23)
> > >>
> > >> The problem, which may be apparent after looking at it, is that
> something like the "combins" function above is not well suited to a long
> series of collections (frankly, for what I plan on using it for, this is not
> a necessary feature--but still). In order to do a combination of + with [0
> 128], [0 64], [0 32], [0 16], [0 8], [0 4], [0 2], and [0 1] (which should
> return a list of all the positive integers that are less than 256), I would
> have to create a function that looked something like:
> > >>
> > >> (defn combins2
> > >> [f c1 c2 c3 c4 c5 c6 c7 c8]
> > >> (for [elem1 c1 elem2 c2 elem3 c3 elem4 c4 elem5 c5 elem6 c6 elem7 c7
> elem8 c8]
> > >> (f elem1 elem2 elem3 elem4 elem5 elem6 elem7 elem8))
> > >>
> > >> (combins + [0 128] [0 64] [0 32] [0 16] [0 8] [0 4] [0 2] [0 1])
> > >>>> (1 2 3 4 5 6 7 8 9 10 ... 250 251 252 253 254 255)
> > >>
> > >> I figured that creating a macro would be a more elegant way to handle
> a theoretically limitless series of collections. A quick introduction to
> macros: macros are functions that generate code as output. When writing a
> macro, you use symbols like `, ~, and ~@. The ` symbol (called the backquote
> symbol) is used a lot like the ' symbol (but they are not the same thing),
> to denote something that should be returned as is, not evaluated. The ~
> symbol (called the unquote symbol) is used to denote something that should
> be evaluated and then that return value should be inserted as is. The ~@
> symbol (called the unqoute splice symbol, or something) is pretty cool, it
> returns the elements inside of what it evaluates to. Both of the unquote
> symbols have to be used inside a backquoted block of code. Here's an example
> of how it works:
> > >>
> > >> (def example-collection [1 2 3])
> > >>
> > >> ~example-collection
> > >>>> #<CompilerException java.lang.IllegalStateException: Var
> clojure.core/unquote is unbound. (NO_SOURCE_FILE:0)>
> > >>
> > >> ~@example-collection
> > >>>> #<CompilerException java.lang.IllegalStateException: Var
> clojure.core/unquote-splicing is unbound. (NO_SOURCE_FILE:0)>
> > >>
> > >> `(list 0 example-collection 4 5 6)
> > >>>> (clojure.core/list 0 user.core/example-collection 4 5 6)
> > >>
> > >> `(list 0 ~example-collection 4 5 6)
> > >>>> (clojure.core/list 0 [1 2 3] 4 5 6)
> > >>
> > >> `(list 0 ~@example-collection 4 5 6)
> > >>>> (clojure.core/list 0 1 2 3 4 5 6)
> > >>
> > >> Note the difference between the above examples versus using ' instead
> of `, which just returns exactly what comes after it.
> > >>
> > >> '(list 0 example-collection 4 5 6)
> > >>>> (list 0 example-collection 4 5 6)
> > >>
> > >> '(list 0 ~example-collection 4 5 6)
> > >>>> (list 0 ~example-collection 4 5 6)
> > >>
> > >> '(list 0 ~@example-collection 4 5 6)
> > >>>> (list 0 ~@example-collection 4 5 6)
> > >>
> > >> In my case I want to generate a "for" with a variable number of
> bindings. Essentially, instead of passing a "hard-coded" vector of variables
> and bindings to the "for" list comprehension, I'm passing a vector that has
> been derived. After a lot (a lot) of attempts, I managed to come up with
> something elegant and concise, using an infinite lazy list of "anonymous"
> symbols (in clojure, I believe it is idiomatic to use an underscore to
> denote a variable that is being defined whose name is not important). For
> people unfamiliar with macros--I'm kind of jumping right into the deep-end
> here, apologies:
> > >>
> > >> (def infinite-symbol-list
> > >> (map #(symbol (format "_%d" %)) (iterate inc 1)))
> > >>
> > >> (take 4 infinite-symbol-list)
> > >>>> (_1 _2 _3 _4)
> > >>
> > >> (defmacro combins
> > >> [f & colls]
> > >> `(for ~(vec (interleave infinite-symbol-list colls))
> > >> (~f ~@(take (count colls) infinite-symbol-list))))
> > >>
> > >> (combins + [0 128] [0 64] [0 32] [0 16] [0 8] [0 4] [0 2] [0 1])
> > >>>> (1 2 3 4 5 6 7 8 9 10 ... 250 251 252 253 254 255)
> > >>
> > >> I'll try to quickly explain the macro I wrote. When you have "&
> var-name" at the end of an argument list for a function, it makes a list out
> of the remaining arguments and stores that in "var-name". For example:
> > >>
> > >> (defn example-fn
> > >> [a & more]
> > >> (list a more))
> > >>
> > >> (example-fn 1 2 3 4 5 6 7 8)
> > >>>> (1 (2 3 4 5 6 7 8))
> > >>
> > >> The "interleave" function takes two collections, and then returns a
> list of (first coll1), (first coll2), (second coll1), (second coll2), etc.
> until one of the collections runs out of elements. Here, I am interleaving
> the infinite symbol list (_1 _2 _3 ...) with the "colls"--which should be a
> collection of the series of collections passed to combins. Let's represent
> colls as (c1 c2 c3 ... cN) where cN is the last collection in the series.
> Interleave these will return (_1 c1 _2 c2 _3 c3... _N cN). Calling "vec" on
> this will return [_1 c1 _2 c2 _3 c3 ... _N cN]. Since ~ will return the
> result of evaluating what is after it, `(for ~(vec (interleave
> infinite-symbol-list colls)) ...) will return (clojure.core/for [_1 c1 _2 c2
> _3 c3 ... _N cN] ...) which is exactly what I want. After this, I want to
> call f using _1 as the first argument, _2 as the second argument, etc. I
> have to do ~f to ensure that I am using "the function inside f" (e.g. "+")
> as opposed to "f itself" (which doe
> > > s not exist). Then, because infinite-symbol-list is infinite (shocker,
> I know) we have to make sure to take only what we need of it. Doing (take
> (count colls) infinite-symbol-list) will return (_1 _2 _3 ... _N), where N
> is the number of collections in the series (i.e. "(count colls)"). Since ~@
> will return the elements inside of the return value of the stuff after it,
> ~@ should return _1 _2 _3 ... _N. So, in the example that f is +, (~f
> ~@(take (count colls) infinite-symbol-list)) will return (+ _1 _2 _3 ...
> _N). Great. Still using + as the example f, the macro as a whole will return
> and then evaluate:
> > >>
> > >> (clojure.core/for [_1 c1 _2 c2 _3 c3 ... _N cN]
> > >> (+ _1 _2 _3 ... _N))
> > >>
> > >> I hope all of that made sense. So...my question. Originally, I told
> you that I was planning on testing each arity of each function, using all
> the possible combinations of argument types, to discover what argument types
> each arity would accept. If I were just trying to test a single list of
> argument types on a single function, I could do this:
> > >>
> > >> (def arg-types
> > >> {\t :character
> > >> [1 2 3 4] :collection
> > >> + :function
> > >> 42 :number
> > >> #"ab" :regex
> > >> #{1 2 3 4} :set
> > >> "Text is in here." :string})
> > >>
> > >> (defn try-args
> > >> [f & elems]
> > >> (try (do (apply f elems) (cons f (map test-types elems)))
> > >> (catch Exception e)))
> > >>
> > >> (try-args + \t)
> > >>>> nil
> > >>
> > >> (try-args + 42)
> > >>>> (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number)
> > >>
> > >> (try-args + 42 \t)
> > >>>> nil
> > >>
> > >> (try-args + 42 42)
> > >>>> (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number :number)
> > >>
> > >> All that #<core$_PLUS...> nonsense is equivalent to +. So it makes
> sense to me that I could do the following to test what argument types a
> 1-arity call to + or - accepts:
> > >>
> > >> (combins try-args [+ -] (keys arg-types))
> > >>>> #<ClassCastException java.lang.ClassCastException:
> java.lang.Character cannot be cast to clojure.lang.IFn>
> > >>
> > >> But I can't, and it looks like it's trying to evaluate the result of
> (keys (arg-types)), and I don't understand why, but if I cast the result
> into a vector that should fix the problem:
> > >>
> > >> (combins try-args [+ -] (vec (keys arg-types))
> > >>>> (nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
> > >>
> > >> So it "works" now, but it's not returning what it "should" be. Here's
> the equivalent of what I expect the macro to return and evaluate:
> > >>
> > >> (for [_1 [+ -] _2 (vec (keys arg-types))]
> > >> (try-args _1 _2))
> > >>>> (nil nil nil (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number)
> nil nil nil nil nil nil (#<core$_ clojure.core$_ at ac9cbe> :number) nil nil
> nil)
> > >>
> > >> Wow, just right now, at this point in writing this email, I just fixed
> the problem. If instead of:
> > >>
> > >> (defmacro combins
> > >> [f & colls]
> > >> `(for ~(vec (interleave infinite-symbol-list colls))
> > >> (~f ~@(take (count colls) infinite-symbol-list))))
> > >>
> > >> I change colls to `~colls inside the interleave:
> > >>
> > >> (defmacro combins
> > >> [f & colls]
> > >> `(for ~(vec (interleave infinite-symbol-list `~colls))
> > >> (~f ~@(take (count colls) infinite-symbol-list))))
> > >>
> > >> (combins try-args [+ -] (keys arg-types))
> > >>>> (nil nil nil (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number)
> nil nil nil nil nil nil (#<core$_ clojure.core$_ at ac9cbe> :number) nil nil
> nil)
> > >>
> > >> It works! Can anybody explain why that works? I just now tried this
> thinking it might solve the problem where it seemed to be evaluating the
> terms in colls. I figured that back-quoting the return value might stop the
> return value from further evaluation. But I have no idea why this solves the
> problem of it returning nils where it should not have been--I still don't
> even know why it was doing that incorrectly in the first place.
> > >>
> > >> -Omri
> > >> _______________________________________________
> > >> Cs254f11 mailing list
> > >> Cs254f11 at lists.hampshire.edu
> > >> https://lists.hampshire.edu/mailman/listinfo/cs254f11
> > >
> > > --
> > > Lee Spector, Professor of Computer Science
> > > Cognitive Science, Hampshire College
> > > 893 West Street, Amherst, MA 01002-3359
> > > lspector at hampshire.edu, http://hampshire.edu/lspector/
> > > Phone: 413-559-5352, Fax: 413-559-5438
> > >
> > > _______________________________________________
> > > Cs254f11 mailing list
> > > Cs254f11 at lists.hampshire.edu
> > > https://lists.hampshire.edu/mailman/listinfo/cs254f11
> >
> > --
> > Lee Spector, Professor of Computer Science
> > Cognitive Science, Hampshire College
> > 893 West Street, Amherst, MA 01002-3359
> > lspector at hampshire.edu, http://hampshire.edu/lspector/
> > Phone: 413-559-5352, Fax: 413-559-5438
> >
> >
> > _______________________________________________
> > Cs254f11 mailing list
> > Cs254f11 at lists.hampshire.edu
> > https://lists.hampshire.edu/mailman/listinfo/cs254f11
>
> --
> Lee Spector, Professor of Computer Science
> Cognitive Science, Hampshire College
> 893 West Street, Amherst, MA 01002-3359
> lspector at hampshire.edu, http://hampshire.edu/lspector/
> Phone: 413-559-5352, Fax: 413-559-5438
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.hampshire.edu/pipermail/cs254f11/attachments/20111012/252db946/attachment-0001.htm>
More information about the Cs254f11
mailing list