okay so I'm the one of the technical writers at puppet labs I was the only one for a little while and when I first got here we had this thing called the language tutorial it wasn't a tutorial it was a big well it was a bit of a mess actually um the you can see that there are additional language features in it it was kind of the whole it was the whole thing all on one page trying to talk about the puppet language and not really organized in the best way and a lot of it
was kind of out of date and that was because the puppet language accreted over a pretty long period of time and change stewardship a couple of times as engineers came and went and so did the docs and the result of that was that the language tutorial was I it might have actually been a tutorial at one point is hard to tell but it wasn't super useful it got the job done but it was it was hard to find what you were looking for so I had a good / bad idea which was to rewrite the
whole thing as a multi-page thing with a kind of visual index with you can see it on the website basically it's the rewritten language reference it works okay um but in order to do that I had to go through the language tutorial and see what i could find in terms of is the information even still accurate and i would start investigating one thing and kind of end up down this rabbit hole ah and keep investigating more things and it it got a little out of tan so here's what I found we're just going to take
you on a safari of interesting things that I learned about puppet so what do you think that's going to output true let's talk about collectors so you know the collectors the angle bracket with the the pipes the there was actually there was a quite exactly there was a lot of confusion about what these things were called down in the code they're called collections but then out elsewhere in the docks people were using collections to refer to classes and defined types so that was a note I end up on collectors but spaceship was what the official
name in the language tutorial was and they're used for collecting virtual resources among other things so given these resources here you've got one with the tag called 11 with the tag called 12 well two tags 13 tags one two and three and if you collect based on the tag that all of them have they'll all get realized that's pretty standard but you'll see that we're doing tag equals equals one what happens if you um yeah so that's that's a non transitive Equality operator will also see that if you um if you try to test equality
against an array you'll see that it it well I'm not quite sure what it's doing there you'll see that all three notifies fired even though only one of them has that exact value and one of them doesn't even have one of those values in the array so it's a little unclear what's going on there how about if we say not equal two of them fire but the first one doesn't yeah so what in conclusion I equality is not equal to equality it seems so yeah what's a let's move on to classes how about a class
class and if you say include class what is that yeah well that's you expect that fail right how about if you say include class with quote marks yeah there was a raging success um it is yeah class is a reserved word but it right there's there's reserved and then there's reserved and like you know yeah okay what is search who here knows what searches there's a function called search in puppet core yeah yeah so here we have a class called parent and we're going to say search parent and then outside that class we're going to
try to declare a define type that is defined inside that class which for reasons because puppet existed before modules existed that's the name of that class is actually parent colon colon does something but we're just going to try and say it does something we're also going to say include whoa that class's name isn't well its parent colon colon whoa and then we're going to try and notice with a variable that's in there too and we see that I it well I can give you the output of this sometime later it just dumps a random name
space into the local namespace so what happens is the that that notice doesn't give you anything because it doesn't attach the scope but it does attach the namespace so all of those class declarations and define type declarations totally work outside of the class yeah that's this is still in puppet 3.2 by the way have you seen this syntax yeah that's that's an array of resource references yeah you can feed that to a meta parameter and make a bunch of relationships and it works have you seen this one how about this one that's actually the same
as the last one I don't really know what's going on there i must flatten it or something they all kind of looks similar so you would kind of try to use them in similar places ah but we can see here that I they mostly work until you try to put an array of resource references on one side of a chaining arrow and that blows up how about this if you if you give a resource some an attribute override block like you would in an inherited class that works with a multi reference like that but you
can't give it an array of resource references so they're completely different syntaxes and I named the non array one multi references because it seemed to be a thing that didn't actually have a name yet Steph is exciting here's another one uh what does this do yeah so we said include bar but it didn't actually include bar that's because include bar actually never means include bar it means try to include a bunch of completely unrelated horse and then include bar as a last resort if none of that works so you'll see that when we were inside
class foo saying include bar tried to include flu colon colon bar ah yeah yeah that's bug 2053 yeah how about case sensitivity the as you can see the Equality operator is actually not case sensitive but the in operator is anyway [Laughter] yeah false mrs. truth you guys have probably this was actually one of the first things I ran into when I was collecting wat was that the facts are all strings so if you yeah and the string false is actually true yeah yeah there's actually a um there's a string to bhool function in standard lib
that you can use to protect yourself that way and as soon as I started doing that it kind of escalated and I started questioning the very meaning of truth [Laughter] the let me see if I can try to remember where some of these actually went I feel like resource references are always true even if the type doesn't exist and the instance doesn't exist negative numbers are true zero is true string to bhool zero is true that was kind of exciting um what else what else or no I think that was only for a no that's
right i was putting these for what would be true if it didn't fail anyway yeah that was that was fun that's it again hub Rico you can run it all yourself no I let's express ourselves with some expressions you've seen this before have you seen this before yet you can just throw random expressions into a variable interpolation token in a string with double quotes and I was so sure that was going to fail I was positive so I went exploring a little more you can put expression tests in there I thought that one was worthy
of some Comic Sans key on his hands what else do we have here the is you can yeah you can use an in statement in there of course if you can do a regular expression test why not so that's pretty cool this one actually blows up you can do expressions with variables and literals in there but you can't do function calls yeah let's go look up well if you if you reference a variable inside one of those expressions it'll come from wherever that variable is supposed to come from so I that could come from top
scope it can come from local scope are you talking about like the scope don't look up that you'll see in a template you would have to know yeah because that function only works inside of a template and we're not in a template right now yeah yeah yeah yes how about a if you do dollar and then dollar inside that that actually works how would you leave off the sigil but you put a bunch of spaces in front that works you can just put a literal string in there too sorry about my last decade I strings
how about this left operand of multiplication is not a number where well no if you collapse the space it will still do that we're just working on some very unfamiliar definitions of number yeah there we go so actually if you quote the number so there now strings [Laughter] that's how I learned about data types in puppet numbers are actually always strings but the if you if you what's back up a second what's really going on here or no that's right here we go let's skip to the jiff okay so the first bear word in an
interpolation block gets silently turned into a variable which one he became dollar twenty and dollar twenty is reserved for ray x capture variables if you for some reason write a regular expression with more than 20 terms in parentheses except we weren't inside a conditional so it was undef so you're trying to do undef times and then a bareword 20 but the second bear word is still 20 or 80 whatever that was I so that was um so that was raring to go and we we went we'll go through a couple of these the I feel
like yeah yeah we already saw most of these you can see that I wouldn't try to make a like a regular expression with 26 terms and that worked great yeah we can talk about aliases a bit who knows about aliases who knows that there's always an alias yeah all right who knows what a name bar is yeah okay so resources have all of these names and they've got one name called a title they've got another name called a name unless it's not called name unless it's called something like path and then they can have other
names called alias that you can use with Elias meta parameter and these things all kind of it's a little unclear what they're all for the the title and the name bar separates that you can have an identity on the target system but then also have a stable identity within the compiler so that if you're trying to refer to a resource that might have a different path on disk on different types of operating system then you can just refer to it in a stable way that makes total sense but you can sort of here will see
that if you make a relationship to a be to an alias of another resource it actually works fine and you can sort of see that when you try to make a relationship to the name bar instead of the title that will still often work as long as you're using the before and require meta parameters if you try to use the chaining arrows instead you actually can't you have to use the title only yeah okay how about realize when you're using the realize function to make a virtual resource happen that no it has to be the
title can't even use the name bar just has to be the title but uh if you use a collector that totally works [Music] yeah all right so what did I learn and how many of I kept under wraps um a fair amount the I did discover a lot of bugs this way and a lot of these are behaviors that we don't necessarily want that's true that's some of them that's not clear whether we should fix and you would have to do it on a major version boundary anyway you're I'll guarantee you someone is using half
of what you saw so far yeah it's always a tricky question um I know that Henrik right now has done a lot of work with the future parser and the the core team is starting to do some preliminary work to build a puppet spec this is maybe times start talking about what I actually managed to make out of all this what I put together the language reference that the kind of reform language reference is it's not a speck and it's not really a guide and it's not really a complete layout of all of the observed
behavior because I didn't want to overwhelm people and I didn't want to guide people into doing things that we're going to go away so it's kind of this neither fish nor fowl of a document where we're trying to talk about what what kind of like the way you would write puppet code if it were a better world right trying trying to lead people in paths that are going to be supportable and stable over the long term and document as much of the weirdness as we have to to tell people to watch out for it and
the what we found is that that it's not really a good foundation for a spec it's a good start but it's not all these things that will come up someone will find something new that and cover in that reference and that I had never seen before like uh what was that one that hammer it came up with where I laughed really really hard I can't remember it um and the question comes up like is this a feature of the language or is this a plug and you can't tell by looking at the language reference because
it's it's trying to help as best you can with you know today and maybe a couple of years in the future so that was that was an interesting experience to learn what the kind of boundaries between documentation in a spec and at the start I thought I was going to come up with something that could be used as a spec that's not the case so I mean what did I learn from doing all of this spelunking I in the larger sense I mean I learned a lot of stuff I learned that if you're dealing with
a system that's in an unknown state because it's created over the years and things have just fallen out of date information wise do science to it there's there's no there's no substitute for getting elbows deep in it and actually finding out what what different things do and long live systems do get weird the other thing I learned is you've got to talk to the old-timers who've been I'm using old timers loosely because puppet goes back to what 2006 talk to people who have been using the system on a day to day basis because even the
weirdest system often works pretty well for people who know their way around it the trick is that you can't really go up to someone and ask hey tell me everything you know about puppets edge cases because they're not they'll freeze up I'll freeze up I freeze up when we get that this talk took me so long to find all that anyway they'll freeze up and it's because you can't when you get into that mindset of communicating in a an attempt to communicate a complete state of a system you just can't do it so you've got
to take your time you've got to get stuff from people over a long period and i think the weird thing i found was that wat brings us together here is my general theory of social capital of wat in engineering cultures if you are trying to discover things about a system in an unknown state and you're working with a lot of people who can't give you a complete info dump of that state just because they're they're not capable of it no one's capable of that sharing the wat that you find is a good way to provoke
other input of what from up people who will just remember something they saw like last year and you give you get him in the mindset of let's talk about all this weird stuff for looking at and it'll just start coming out so these conversations in the hallway that I had over the course of this long period in 2012 when I was doing this we're really illuminating I would bring up something like I i think that the truth of falsity was the first one that came to me from someone else I was like whoa really and
I talked to someone else about that and they were like oh yeah have you seen this other thing that does something completely weird and then i was told i think that the variable interpolation thing with throwing random expressions in there came from somewhere in there also we're like they're actually like i don't know if this can do this can this do this and i was like let's find out and it's it's a it's this kind of virtuous cycle of talking about talking about investigation that i find roles on really really well if you get a
lot of people contributing just a little bit of what and venting about it can keep your morale up because that's actually pretty funny and because if you're trying to investigate a system in an unknown state it can maybe take a while and you're going to be at it for a bit so take your joy where you can and enjoy yourself as much as you can yeah that's pretty much what i've got I'd love to stick around here for a minute and talk some more about just the process of investigating the language and questions of where
the language might go thank you very much [Music] so yeah just does anyone have something they want to talk about well given that a lot of what you saw is inappropriately exposed Ruby behavior I'm going to say a fair amount of that would still be there no it was more that the remind myself of the state of that Ruby dsl the the Ruby dsl just did odd things because you were loading it as from Ruby code when you're when you're loading puppet code off the disk there's a a sort of predictable path that it goes
through in the system there's a system in place for reloading puppet code there's a system replace for loading puppet code differently from modules if the environment is different on what we found with the ruby dsl was because you are handing it off to the Ruby loader and letting the Ruby loader just do its thing with it you you get things like it'll just stick around when it comes from one environment and it'll never be replaced by something from another environment the environment separation was a really big problem you'll get he trying to remember what some
of the other problems were I feel like the yeah that was that was the thing like it was really sticky right if he wanted to reload a ruby manifest you had to start restart the whole puppetmaster process and that was it was really hard to get around we would have had to build a third code loading system I think to get around that I think and that when we were investigating it well we ultimately fine i don't know if you've been to the i don't think that's happened yet henrik some talk about the future of
the language when we were doing user testing what we often found was that people really wanted to do loops and aeration and collection over a structured data and that was kind of the only thing that really needed three bsl-4 like there were some really edge edge edge cases where nothing but pure Ruby would do but for like ninety-five percent of what people wanted the ninety-five percent of the reasons people wanted the Ruby dsl to work correctly was because they wanted to iterate over at data structure and generate resources from it in a predictable and like
very terse way and so we've been working on just adding that to the puppet language instead as for the general question of whether making a DSL kind of Doom's you did this kind of stuff thank you I don't know if I have the answer to that I think languages are tricky things and they'll always have growing pains but i think the have having a DSL that expresses your native objects in a much more terse and much more friendly way than trying to stuff it into a different language often pays off in ways that are hard
to measure i hope that helps a little let's get a debugging pop again i was using puppet apply for all of this and i was using a lot of its it was very primitive debugging because i am a very primitive debugger I just I've my engineering jobs earn all up to speed at the moment and so it's the equivalent of adding a whole bunch of breakpoints adding a whole bunch of print statements and just kind of let me a run there [Music] oh really I'm glad I oh that's cool nice you were going to ask
something so that sounds to me like something that the future parser will probably take care of because they're the expression based grammars a lot more flexible right now hashes are weird objects in pop it there I'm trying to remember how it works like they're not there they're technically expressions rather than data and that just results in all kinds of weird edge cases yeah I I feel like someone told me that there was a similar issue that was going to go away in the future parser yeah so it's um having watched henrik attack one of those
issues before in the future parser i will say that it's probably easy to fix in the future parser and probably devilishly tough to fix in the current parser ya know yeah keep the noise all about that it'll probably get fixed hey real data types and structured data um yeah we really want to do that and the plan is a little fuzzy to me at the moment the problem with structured facts is that it has this kind of like knock-on effects throughout the entire ecosystem so anything that had been configured to store facts it's going to
start getting some unexpected stuff anything that's been configured to read facts when the cash on disk is going to get some weird stuff anything that reads facts from a fact store like Papa TV is going to start getting weird stuff and I we were experimenting with ideas of having some kind of duel format facts where they could have a structured format and then I string ified format and depending on what interface you were using you'd get different ones so for example within the puppet dsl you would have a top level variable called facts that would
be a hash that would be a big ol data structure containing all facts with their actual data types including structure facts like arrays and hashes I don't currently know the status of that I know it's on the road map I would have to check I mean factories of this recurrence whatever in factories already in public beta 3 there's to say string my act schools [Music] so so that we will actually get a start time then my left but that still leaves of all as top scope variable so whatever their value is right that still leaves
in this top scope failure but it doesn't do that fact sash yeah oh yeah actually and that's something that I've run into before is that the string dual function will actually barf if you feed it a real bull so the pattern that i started using yeah exactly so the pattern i started using is vy of a variable coming in like from a class parameter where I don't know whether it's going to be a string rule or real bull is I will say string to bhool parenthesis quote the variable name so I'll turn whatever it is
into a string and then turn it back in whoville and that worse that's super reliable well I am because I salt I mean like it okay it was a real problem sorry is there an anymore yeah it's part of puppy No all right cool yeah it's non-sterile again anything else what's going away in the future parser and puppet for I don't know yet I would like to see a lot of that funny stuff go away the part of why they are starting to talk about a spec now is that imposing a speck on the puppet
language as a stance in practice would be really really tough but since we're going to be making sort of a clean break at a major version boundary where we throw over to a future parser anyway we could say the future parser is something that works with an actual spec right we can kind of start from the ground up not not from the ground up but in a sort of do a much more controlled evolution the language from that point on and since I believe the future parser is going to stay disabled in puppet for even
though we're going to turn it to default that'll give everyone a little while to get their stuff in order and figure out how to live with the new world yeah kind of getting started with puppet on Windows um we would like to there yeah there's there's always more work than the documentation team can do I would be excited about doing that I don't know if we're going to get time to do it soon but I do know that expanding further into Windows is pretty strategically important to us so yeah let's just say that I would
really like to and if you can tell me more about what kind of requirements you're running into as you're trying to teach these windows people puppet that might be helpful whether you think like something basically the same as learning puppet with just more windows examples would help or does that mean or whether you think a more serious overhaul is needed that's something where I haven't haven't had to try to teach a bunch of Windows people how to use public before so I could really use some input on that all right wonderful how do you mean
completely yeah yeah and like the things like the scheduled task instead of chron and stuff like that it does make it a little bit hard um as far as I can tell it's kind of unaffordable yeah Spencer we're trying to go a windows second into that right now I got no screen capturing of solid yeah this is starting something like maybe we should do a kind of it's going to work session later on yeah cool looks like we've got a six minutes left so let's go ahead and wrap it up I think thanks for coming
out to my talk I really appreciate it it was really great to be able to share all the weird stuff I found with you and there's a repo on github that has all of my test manifests in it it's called evil made manifest and yeah I accept only the worst full requests yeah thanks for coming down [Applause]