[Cs254f11] Macro confusion

Tue Oct 11 22:45:38 EDT 2011

Thanks Lee! Writing macros does feel quite like an art, and I seem to figure
it out more with trial and error than with formal logic.

Yeah, I already had a look at clojure.contrib.combinatorics. It has a lot of
cool stuff, but not anything that completely did what I was trying to do, so
I figured I'd make my own. And, as you said, it's a good learning
experience.

I did a macro so that I could pass the "for" a theoretically limitless
vector. I could just changed the "defmacro" to a "defn", but then I would
just have to evaluate what it returns. If I could come up with an
alternative for using "for" as the core function, then I could probably do
it as a function. Actually, I was originally trying to use "map", but I
wasn't successful in getting past a combins function that could handle any
more than two collections:

(defn combins
  [f c1 c2]
    (map (fn [n] (map (partial f n) c2)) c1)

I did a mental back-flip just getting this far, and I'm struggling with how
to even add a third collection to the series, let alone generalize it to [f
& colls]. Any thoughts?

As for the errors I got, I see why my fix solved the "eager evaluation"
problem that was happening. As you said, it was trying to call \t as a
function. What I don't understand is what the problem was when I originally
tried casting the argument as a vector--which should evaluate to itself and
not call \t as a function--which it didn't. However, it doesn't return the
correct answer:

(combins try-args [+ -] (vec (keys arg-types))
>> (nil nil nil nil nil nil nil nil nil nil nil nil nil nil)

When I did that fix, it didn't "error", but it did give me an incorrect
result. I don't understand why it was giving me incorrect results, and I
don't understand why doing `~colls instead of colls changed that.

-Omri

On Tue, Oct 11, 2011 at 4:50 PM, Lee Spector <lspector at hampshire.edu> wrote:

>
> One other thing that may be of interest: clojure.contrib.combinatorics
>
>  -Lee
>
>
> On Oct 11, 2011, at 4:22 PM, Lee Spector wrote:
>
> >
> > Lots of good stuff in there!
> >
> > A few comments:
> >
> > - Is there really any reason to make this a macro? Why not just a
> function that takes the same "& colls" argument? As far as I can tell that
> would work just as well in this case and be less confusing. On the other
> hand, you got some good practice writing macros :-). But on the third hand
> you should only use macros when you really need them -- which is usually
> when you want to control whether code passed in as arguments gets evaluated
> (or how many times it gets evaluated). One of the downsides of writing
> macros when functions could do the job is that you can pass a function to
> another function, map it down a sequence, etc., but you can't do any such
> thing with a macro.
> >
> > - You may well be able to get the argument/type information that you're
> computing here from Java "reflection" rather than trying things and catching
> exceptions. I'm not sure, and again what you're doing is a cool exercise in
> any event, but you might want to look into this.
> >
> > - Writing macros is an art, and a pretty deep one. Paul Graham's book "On
> Lisp" is largely about the art of macro writing, and I think it's extremely
> cool. But it uses Common Lisp rather than Clojure, so some but not all of it
> will apply to Clojure.
> >
> > - In my macro-writing past, which was mostly in Common Lisp, I sometimes
> ended up doing the same kind of trick that you did with backquote-tilde,
> although it would have been backquote-comma in common lisp. Generally this
> happens when you want the macro's argument to be evaluated, but then you
> want that value to be quoted in the macro's expansion code. In your case
> evaluating the argument to the macro call will give (\t [1 2 3 4] + :42
> #"ab" #{1 2 3 4} "Text is in here.") and you want that to be quoted in the
> macro expansion's call to interleave. If it isn't quoted there then when the
> expansion is evaluated it will try to call \t as a function, which gives the
> error that you saw.
> >
> > I think that that last point answers your question, but I also think that
> you should look into the other points, particularly the first.
> >
> > -Lee
> >
> >
> > On Oct 11, 2011, at 3:50 PM, Omri Bernstein wrote:
> >
> >> As a presage to this email, I spend a lot of time explaining, and my
> question doesn't really come until the very end. Also, by the time I got to
> the end of writing , I managed to accidentally solve my original question,
> and now I'm curious whether anybody can tell me why what I did solved the
> problem.
> >>
> >> I've been trying to work on getting the arities for all the clojure
> functions, and what types of arguments each arity for each function will
> accept. I'm on the second part now, and I plan on going about it by trying
> all the possible combinations of argument types, and returning the types of
> the ones that don't error. So I decided to make a general-purpose macro for
> returning a list of all the possible combinations of applying a function f
> to a series of collections--using each value from the first collection as
> the first argument to f, and each from the second collection as the second
> argument to f, etc. If you understand how the "for" loop comprehension works
> (if not, maybe the examples here would help), this is an example of how the
> concept would work, using + as the function, [1 2 3] as the first
> collection, and [10 20] as the second collection:
> >>
> >> (for [elem1 [1 2 3] elem2 [10 20]]
> >>  (+ elem1 elem2))
> >>>> (11 21 12 22 13 23)
> >>
> >> This could be generalized to:
> >>
> >> (defn combins
> >>  [f c1 c2]
> >>  (for [elem1 c1 elem2 c2]
> >>    (f elem1 elem2))
> >>
> >> (combins + [1 2 3] [10 20])
> >>>> (11 21 12 22 13 23)
> >>
> >> The problem, which may be apparent after looking at it, is that
> something like the "combins" function above is not well suited to a long
> series of collections (frankly, for what I plan on using it for, this is not
> a necessary feature--but still). In order to do a combination of + with [0
> 128], [0 64], [0 32], [0 16], [0 8], [0 4], [0 2], and [0 1] (which should
> return a list of all the positive integers that are less than 256), I would
> have to create a function that looked something like:
> >>
> >> (defn combins2
> >>  [f c1 c2 c3 c4 c5 c6 c7 c8]
> >>  (for [elem1 c1 elem2 c2 elem3 c3 elem4 c4 elem5 c5 elem6 c6 elem7 c7
> elem8 c8]
> >>    (f elem1 elem2 elem3 elem4 elem5 elem6 elem7 elem8))
> >>
> >> (combins + [0 128] [0 64] [0 32] [0 16] [0 8] [0 4] [0 2] [0 1])
> >>>> (1 2 3 4 5 6 7 8 9 10 ... 250 251 252 253 254 255)
> >>
> >> I figured that creating a macro would be a more elegant way to handle a
> theoretically limitless series of collections. A quick introduction to
> macros: macros are functions that generate code as output. When writing a
> macro, you use symbols like `, ~, and ~@. The ` symbol (called the backquote
> symbol) is used a lot like the ' symbol (but they are not the same thing),
> to denote something that should be returned as is, not evaluated. The ~
> symbol (called the unquote symbol) is used to denote something that should
> be evaluated and then that return value should be inserted as is. The ~@
> symbol (called the unqoute splice symbol, or something) is pretty cool, it
> returns the elements inside of what it evaluates to. Both of the unquote
> symbols have to be used inside a backquoted block of code. Here's an example
> of how it works:
> >>
> >> (def example-collection [1 2 3])
> >>
> >> ~example-collection
> >>>> #<CompilerException java.lang.IllegalStateException: Var
> clojure.core/unquote is unbound. (NO_SOURCE_FILE:0)>
> >>
> >> ~@example-collection
> >>>> #<CompilerException java.lang.IllegalStateException: Var
> clojure.core/unquote-splicing is unbound. (NO_SOURCE_FILE:0)>
> >>
> >> `(list 0 example-collection 4 5 6)
> >>>> (clojure.core/list 0 user.core/example-collection 4 5 6)
> >>
> >> `(list 0 ~example-collection 4 5 6)
> >>>> (clojure.core/list 0 [1 2 3] 4 5 6)
> >>
> >> `(list 0 ~@example-collection 4 5 6)
> >>>> (clojure.core/list 0 1 2 3 4 5 6)
> >>
> >> Note the difference between the above examples versus using ' instead of
> `, which just returns exactly what comes after it.
> >>
> >> '(list 0 example-collection 4 5 6)
> >>>> (list 0 example-collection 4 5 6)
> >>
> >> '(list 0 ~example-collection 4 5 6)
> >>>> (list 0 ~example-collection 4 5 6)
> >>
> >> '(list 0 ~@example-collection 4 5 6)
> >>>> (list 0 ~@example-collection 4 5 6)
> >>
> >> In my case I want to generate a "for" with a variable number of
> bindings. Essentially, instead of passing a "hard-coded" vector of variables
> and bindings to the "for" list comprehension, I'm passing a vector that has
> been derived. After a lot (a lot) of attempts, I managed to come up with
> something elegant and concise, using an infinite lazy list of "anonymous"
> symbols (in clojure, I believe it is idiomatic to use an underscore to
> denote a variable that is being defined whose name is not important). For
> people unfamiliar with macros--I'm kind of jumping right into the deep-end
> here, apologies:
> >>
> >> (def infinite-symbol-list
> >>  (map #(symbol (format "_%d" %)) (iterate inc 1)))
> >>
> >> (take 4 infinite-symbol-list)
> >>>> (_1 _2 _3 _4)
> >>
> >> (defmacro combins
> >>  [f & colls]
> >>  `(for ~(vec (interleave infinite-symbol-list colls))
> >>     (~f ~@(take (count colls) infinite-symbol-list))))
> >>
> >> (combins + [0 128] [0 64] [0 32] [0 16] [0 8] [0 4] [0 2] [0 1])
> >>>> (1 2 3 4 5 6 7 8 9 10 ... 250 251 252 253 254 255)
> >>
> >> I'll try to quickly explain the macro I wrote. When you have "&
> var-name" at the end of an argument list for a function, it makes a list out
> of the remaining arguments and stores that in "var-name". For example:
> >>
> >> (defn example-fn
> >>  [a & more]
> >>  (list a more))
> >>
> >> (example-fn 1 2 3 4 5 6 7 8)
> >>>> (1 (2 3 4 5 6 7 8))
> >>
> >> The "interleave" function takes two collections, and then returns a list
> of (first coll1), (first coll2), (second coll1), (second coll2), etc. until
> one of the collections runs out of elements. Here, I am interleaving the
> infinite symbol list (_1 _2 _3 ...) with the "colls"--which should be a
> collection of the series of collections passed to combins. Let's represent
> colls as (c1 c2 c3 ... cN) where cN is the last collection in the series.
> Interleave these will return (_1 c1 _2 c2 _3 c3... _N cN). Calling "vec" on
> this will return [_1 c1 _2 c2 _3 c3 ... _N cN]. Since ~ will return the
> result of evaluating what is after it, `(for ~(vec (interleave
> infinite-symbol-list colls)) ...) will return (clojure.core/for [_1 c1 _2 c2
> _3 c3 ... _N cN] ...) which is exactly what I want. After this, I want to
> call f using _1 as the first argument, _2 as the second argument, etc. I
> have to do ~f to ensure that I am using "the function inside f" (e.g. "+")
> as opposed to "f itself" (which doe
> > s not exist). Then, because infinite-symbol-list is infinite (shocker, I
> know) we have to make sure to take only what we need of it. Doing (take
> (count colls) infinite-symbol-list) will return (_1 _2 _3 ... _N), where N
> is the number of collections in the series (i.e. "(count colls)"). Since ~@
> will return the elements inside of the return value of the stuff after it,
> ~@ should return _1 _2 _3 ... _N. So, in the example that f is +, (~f
> ~@(take (count colls) infinite-symbol-list)) will return (+ _1 _2 _3 ...
> _N). Great. Still using + as the example f, the macro as a whole will return
> and then evaluate:
> >>
> >> (clojure.core/for [_1 c1 _2 c2 _3 c3 ... _N cN]
> >>  (+ _1 _2 _3 ... _N))
> >>
> >> I hope all of that made sense. So...my question. Originally, I told you
> that I was planning on testing each arity of each function, using all the
> possible combinations of argument types, to discover what argument types
> each arity would accept. If I were just trying to test a single list of
> argument types on a single function, I could do this:
> >>
> >> (def arg-types
> >>  {\t :character
> >>   [1 2 3 4] :collection
> >>   + :function
> >>   42 :number
> >>   #"ab" :regex
> >>   #{1 2 3 4} :set
> >>   "Text is in here." :string})
> >>
> >> (defn try-args
> >>  [f & elems]
> >>  (try (do (apply f elems) (cons f (map test-types elems)))
> >>       (catch Exception e)))
> >>
> >> (try-args + \t)
> >>>> nil
> >>
> >> (try-args + 42)
> >>>> (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number)
> >>
> >> (try-args + 42 \t)
> >>>> nil
> >>
> >> (try-args + 42 42)
> >>>> (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number :number)
> >>
> >> All that #<core$_PLUS...> nonsense is equivalent to +. So it makes sense
> to me that I could do the following to test what argument types a 1-arity
> call to + or - accepts:
> >>
> >> (combins try-args [+ -] (keys arg-types))
> >>>> #<ClassCastException java.lang.ClassCastException: java.lang.Character
> cannot be cast to clojure.lang.IFn>
> >>
> >> But I can't, and it looks like it's trying to evaluate the result of
> (keys (arg-types)), and I don't understand why, but if I cast the result
> into a vector that should fix the problem:
> >>
> >> (combins try-args [+ -] (vec (keys arg-types))
> >>>> (nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
> >>
> >> So it "works" now, but it's not returning what it "should" be. Here's
> the equivalent of what I expect the macro to return and evaluate:
> >>
> >> (for [_1 [+ -] _2 (vec (keys arg-types))]
> >>  (try-args _1 _2))
> >>>> (nil nil nil (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number) nil
> nil nil nil nil nil (#<core$_ clojure.core$_ at ac9cbe> :number) nil nil nil)
> >>
> >> Wow, just right now, at this point in writing this email, I just fixed
> the problem. If instead of:
> >>
> >> (defmacro combins
> >>  [f & colls]
> >>  `(for ~(vec (interleave infinite-symbol-list colls))
> >>     (~f ~@(take (count colls) infinite-symbol-list))))
> >>
> >> I change colls to `~colls inside the interleave:
> >>
> >> (defmacro combins
> >>  [f & colls]
> >>  `(for ~(vec (interleave infinite-symbol-list `~colls))
> >>     (~f ~@(take (count colls) infinite-symbol-list))))
> >>
> >> (combins try-args [+ -] (keys arg-types))
> >>>> (nil nil nil (#<core$_PLUS_ clojure.core$_PLUS_ at 507726> :number) nil
> nil nil nil nil nil (#<core$_ clojure.core$_ at ac9cbe> :number) nil nil nil)
> >>
> >> It works! Can anybody explain why that works? I just now tried this
> thinking it might solve the problem where it seemed to be evaluating the
> terms in colls. I figured that back-quoting the return value might stop the
> return value from further evaluation. But I have no idea why this solves the
> problem of it returning nils where it should not have been--I still don't
> even know why it was doing that incorrectly in the first place.
> >>
> >> -Omri
> >> _______________________________________________
> >> Cs254f11 mailing list
> >> Cs254f11 at lists.hampshire.edu
> >> https://lists.hampshire.edu/mailman/listinfo/cs254f11
> >
> > --
> > Lee Spector, Professor of Computer Science
> > Cognitive Science, Hampshire College
> > 893 West Street, Amherst, MA 01002-3359
> > lspector at hampshire.edu, http://hampshire.edu/lspector/
> > Phone: 413-559-5352, Fax: 413-559-5438
> >
> > _______________________________________________
> > Cs254f11 mailing list
> > Cs254f11 at lists.hampshire.edu
> > https://lists.hampshire.edu/mailman/listinfo/cs254f11
>
> --
> Lee Spector, Professor of Computer Science
> Cognitive Science, Hampshire College
> 893 West Street, Amherst, MA 01002-3359
> lspector at hampshire.edu, http://hampshire.edu/lspector/
> Phone: 413-559-5352, Fax: 413-559-5438
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.hampshire.edu/pipermail/cs254f11/attachments/20111011/1829b016/attachment-0001.htm>