In electronic music equipment and software, what are some good ways of cutting costs without sacrificing quality/effort?

Question: I'm interested in getting into producing electronic music (EDM, house, whatever I end up digging at the time), but I've recently run into a roadblock: money.
Potential costs include:

Computer
DAW
Keyboard/Synth
VSTs
High quality sample packs
Purchasing existing music for samples and inspiration

What are some good ways of cutting costs without sacrificing quality/effort?
Answer:
Computer: You're using one right now. You don't need a fancy new computer to make music.
DAW: There are many free DAWs. REAPer is free if you don't mind a nag screen, Audacity is free and open source.
Keyboard/Synth: You can get an entry keyboard for $100 that will work fine.
VSTs: There are many free VSTs available from plenty of websites.
Samples: Free on many sites as well.

You don't need top of the line equipment to get started. Making music has always been a "DIY" activity, and many great musicians and producers have created amazing material with just the gear that was lying around.
Personally, I got started using free or dirt cheap, second hand products, and you upgrade as you go along, getting gear that you know that you'll need and use and will make your life easier. And you won't know what you'll need and use unless you cut your teeth using what you've got on hand.
Comment: I would argue that you don't need an external keyboard at all to start out. I spent at least a year making music with just a laptop, before gradually moving on to more serious equipment and more demanding software.
Comment: On the other hand, you can get an M-Audio Keyrig 49 (what I use) on ebay for ~$50. It's not fancy, but it's small, and 49 keys is a good amount. IMO having a keyboard is incredibly useful in composition.
Comment: I'm just learning as well.  I got audacity and a virtual keyboard.  I'm buying old obscure records from second hand and used record shops that are no longer covered by copyrights looking for samples and inspiration.  I'm going to check out reaper, from these posts, it seems to be good entry level software.
Comment: Renoise is also worth a mention alongside REAPER. Free if you don't mind (very liberal) demo-mode restrictions, can use both VSTs and samples, and can be "played" with the PC keyboard if you don't have a MIDI controller.
Answer: It doesn't need to be expensive.
Computer: You don't need an expensive system, and chances are that the one you are using right now is more than enough. I have used a 2GB RAM, 2.4 GHz Intel Core 2 Duo system for production (including mixing). The more powerful your system is, the bigger your real-time toolbox is: more channels, more effects, more programs, etc. This is awesome, but you can get around the limits of a cheap system by using techniques like freezing tracks, that is nothing more than recording a track with all its processes, to free that memory for something else. You might never need to do that as a beginner, though.
Audio Synthesis Environments: We have Reaktor and Max/MSP as the commercial alternatives, both at $400 USD. Both are great, but we also have free alternatives that are as good: Pure Data (from the creator of Max/MSP, both are very similar), SuperCollider, Csound, and Jeskola Buzz.
DAW: Some free DAWs worth checking out: LMMS, Ardour, Reaper (demo version has no restrictions), and Audiotool (browser-based, not very robust but decent enough to learn the basics).
Two mid-price-range DAWs are Renoise, and Logic Pro (which might be the best price-quality ratio you will find).
Other famous, awesome, expensive DAWs have cheap, restricted, versions. Like Reason, Cubase, and Ableton Live.
The barrier gets a little higher when talking about non-linear DAWs, but you still have cheap options. Ableton Live has a cheap intro version at $100 USD, but it is very limiting. The complete version of Ableton Live is much more expensive (worth every penny in my opinion, but don't buy it unless you know what you are doing). Bitwig Studio is another alternative that is around 50% cheaper than the complete version of Ableton Live, without the limitations of the intro version.
Plugins (VST, AU, AAX, etc): Your DAW will already include many tools. If you want to go beyond that, the free plugins community is huge. There are so many free plugins that you'll struggle to find the good quality ones. My suggestion here is to experiment with what you can find (who knows what undiscovered gems you could find?) but also use forums and lists to guide your search. There are well-known free synths/fx/whatever that have proven to be great and solid in the eyes and ears of many people.
One famous example is Synth1 (don't judge it by its ugly GUI). One good place to start your search in is KVR Audio.
High quality sample packs: Make those samples yourself! Make your kicks, your snares, your hats: you'll find in sound design a new dimension of expressiveness, carving exactly what you want, as opposed to what you could find. (Biased opinion alert here, big fan of sound design).
I know sometimes samples can be useful or necessary, depending on what you have in mind. Internet has you covered on this one too. Google-fu will lead you to a huge array of sample options, of all the quality range. Use your ears to filter out the low quality stuff, noise being the most common offender.
Keyboard and/or other controllers: You don't need it. You'll have other means of control. You don't want to make choices regarding hardware controllers until you know where you are going, and that's not happening on the first weeks. Why? Because on this subject you get what you payed for. Cheap hardware will tend to fail and suck more than expensive hardware. There is no shortcut here.
There are some exceptions, but they are rare. Some I can think of: Behringer B-Control Rotary BCR2000 (rotary controls), Korg nanoKEY2 (keyboardish), Korg microKEY25 (keyboard). Don't buy hardware until you need it though, that means that you understand why and how you will implement it in your production or execution. You might end up with things that you will never use, which is very common since there are so many shiny toys out there.
Hardware synths: Stay away from these until you know what you are doing (which again, won't happen in the first weeks). You have more than enough to play with in your software synths, enough to learn everything about all types of synthesis. Hardware synths are very expensive.
I won't say that you don't need them, since there are dynamics that are still very hard or impossible to replicate in software, so you might go for a specific timbre of a specific synth, but I will say that you don't need them now that you are still struggling with the basics. Forget about them for now.
Comment: Great answer! It can't be stressed enough that the important thing is to get started with what is available, in order to find out what fits your individual style. Or to find out that this area wasn't as interesting as it first seemed.
Comment: The means of music production are incredibly cheap, which is why so many people are doing it in their bedrooms.
Answer: Expensive compared to what? If you looked at learning sax, guitar, piano etc., and bought new, as you appear determined to do, you would shell out a load of money with those.Especially electric guitar, because you would feel you need a decent amp., then effects, et al.
When you learned to ride a bike, you hopefully didn't go and buy a $3000 racing road bike, so why spend loads on this?
Buying pre-used may have some disadvantages in that it may be pre-abused. However, if you find someone who knows what he's up to, he will help with the selection.The gear may be not top spec. or bang up-to-date, but it will get you started. The computer doesn't need to be all singing all dancing, as long as it's powerful enough to do its job.Even a brand new one will be out of date too soon. Let someone else take the drop in value! There are free packages out there.There are pieces of equipment that 'doting dad' has bought for 'spoilt child' that are languishing, waiting for you to buy.
Once into it, even if you bought new, you'd realise that some of the stuff is not what you need, but you won't know till you get started. Good luck in your search !
Answer: Well, if you're into commercial production within the music industry, it's pretty expensive. A hobby artist does not need expensive equipment at all.
Your focus should be on creating the music, not the bit rate of your sample packs, save that for a re-mix/re-master, let the quality evolve.
There are plenty of free resources to play with:

http://www.studiotoolz.net/

You could even do everything in a browser: (!!)

http://www.audiotool.com/
Answer: If you're up for using Linux, check out Ardour.
It is an open source DAW, and costs 1$ or more (whatever you want to pay), so I guess it technically doesn't fall into the "those garbage free programs" category.  ;-)  It has been used by professional musicians and wannabes like me for years, and I'm very happy with the results.
Just go check it out...
Comment: Ardour is *really* good for studio musicians and others mostly focussed on audio-based editing. It's definitely usable for electronic music too (I've made a few tracks with it) but the MIDI editing UI can be a bit annoying. Still, by far the best open source option available. You can install it for free from the Ubuntu repositories, but if you like it I'd definitely recommend donating some cash if you can.
Comment: Actually, I currently have my computer partitioned with Ubuntu and Windows. And I do like the sound of $1! Would you say Ardour is worth taking the time to learn, even with the other suggested options (e.g. reaper) available?
Comment: Absolutely.  I found it very intuitive, even coming from my non-professional background.  From what I hear though, there are a number of people who have found it very easy to transition from ProTools to Ardour, so I'm guessing the interface is similar.
Answer: Firstly - you need to recognize that actually - you're pretty privileged to live in this day and age of electronic music. Creating electronic music is far cheaper and more accessible than it ever used to be! ie. You can create electronic music with the computer you already own, I recommend a pair of decent speakers (more below), and a DAW.
This is far cheaper than what you would have previously needed, a mixer, speakers, hardware synth, sequencer etc.
What you actually need to get started
Computer - Presumably, you already own one. You really don't need a high spec computer to get started. Remember people were creating music on computers 10 years ago, and their specs were far less than what you can get for second-hand for $200 today.
When you start out, you're probably not going to be pushing your computer as hard as when you've got a lot of experience anyway.
If you do encounter performance issues, then you can start bouncing your tracks down to improve performance.
DAW - Arguably the biggest barrier to entry, the likes of Ableton costs in the $500 range.
One option here is trying to pick up a Ableton Lite license, which are often given away with hardware.
A lot of people here are recommending using free DAWs like Reaper. I don't necessarily recommend this path. If Reaper works for you, then great, but IMO some DAWs like Ableton or FL Studio are much easier and better to work with, so you might as well use it from the get go. (Analogously, one can always use Open Office for writing and spreadsheets, but if you're using it everyday, I would definitely recommend using Microsoft Office, it's a far superior product).
Some kind of speakers or headphones - Anything to output sound with, some headphones that you already own, or some home speakers. However, I recommend upgrading these ASAP.
So that's all you actually need to start making sound.
After that
The first thing I'd recommend is DECENT SPEAKERS.
This actually is kind of necessary.
You can learn your DAW without decent speakers, but actually getting your sound right, you really need some decent speakers to hear what you're doing. I recommend buying something like the KRK Rokits, (or comparable speakers) for your entry level. These are decent speakers, and they last, you won't need to replace them.
I definitely recommend buying speakers before you buy any other gear, like MIDI controllers, sound cards, or VSTs.
Once you've got your computer, DAW, and decent speakers, then you really are sorted to make some decent music. There are plenty of free VSTs and samples you can use. If you're using the Ableton, the native VSTs are AWESOME.
As far as a midi controller goes - it's not strictly necessary to make music, but it kind of depends on your workflow, some people work better with hardware than others.
Midi controller - Ok, maybe you want some knobs to play around with. You don't need some full package to play around with straight away, all you need is some knobs. You should be able to buy a second hand MIDI controller for pretty cheap, or maybe pick up one of the Korg Nano series, they're cheap.
VSTs - Not necessary. Use the ones that come with your DAW, and there are plenty of free ones.
Audio interface - Really not necessary to start. Your onboard sound should be fine to work with.
Bottom line:
All you need to get started making decent music is a computer, DAW, and decent speakers.
Definitely prioritise speakers ahead of any MIDI controllers or VSTs.
Answer: If you start out with trackers, there are free options. Renoise is commercial, but is used by e.g. Venetian Snares. That shows it can produce quite good results... There is a trial version of Renoise, but there are also other free trackers, see http://en.wikipedia.org/wiki/Tracker_%28music_software%29.
I would say electronic music is easier to work yourself up in, starting from 0, and still getting quite good results, compared to most other styles.
Answer: See Matt's comment above. There are free DAWs and there are also DAWs that don't cost $750. If you're on a Mac, Logic Pro X is excellent and only costs $200. If you're on a windows, Fruity Loops is $99 for the starter edition which is fully featured, and should be more than sufficient for electronic music. Some of the higher models of Fruity loops also have the ability to record audio if that's something that your electronic music requires.
You also do not need a top of the line computer. If the computer that you're on is too weak, and you need a new one, go for a Windows PC. I've used both Windows and Macs and I cannot say that Macs are superior. So anyone who tells you that you 'need' a Mac, is misleading you. You only need a Mac if you're going to opt for Logic Pro X, which again is a fantastic program. PCs are cheaper, and for the price of a low end Mac you can get a much stronger PC. You can get a a very decent PC that is worthy of recording and producing music for $700.
Keyboard synth, you can get a used keyboard for much less than $500. I suggest an 88 key keyboard. Look at Craigslist or ebay.
VST's can be had for much less than $300 again. If you buy Logic Pro X it comes with a pretty decent collection of VSTs. So you're getting a DAW + very functional and decent VSTs for $200. Fruity Loops also comes with VST's. Again you're getting a DAW + functional VSTs for $99-$300, depending on which choice you go with.
Comment: I don't think you even need to spend $700 on a computer for producing music. I started using Ableton on Windows XP with a single core P4, with maybe 4GB RAM, and it was fine. Eventually it started having performance issues, but you could just bounce the audio down.  You'd pick that computer up for $100 these days.
Answer: I have to say that I agree with others in that you can get started for very little money. As others have said, REAPER is good (though not free, it is fully functioning and unrestricted for your trial use, then only $60 for a personal use license if you decide to buy). I have also found that there are several good forums for virtual software instruments that often have great posts about free instruments and synths. Although I have several really good paid VST instruments, I find myself often using Crystal and Synth1, both of which are free. I use them because they can sound REALLY good. They are both great creative tools that often help me turn noodling around on the keyboard into something I really like and can build a song around.
I saw a few videos of some good musicians playing very cheap instruments (a drummer and a guitarist). The instruments were in bad shape, but the skill of those musicians still resulted in some pretty great sounding music. I realized that it's not really the tools you use, so much as it is the skill you have and develop in yourself. You don't need expensive tools, you just need to go and make your music.
EDIT:
I have recently started using Soundtrap, which is a web-based (Google Chrome) DAW. It includes samples, and a few soft synths and can legitimately be used for free. If you want more of the Soundtrap features, you can pay for a subscription which you can cancel at any time. I have to say that after spending a few weeks with it, I'm really impressed. This is by far the best music collaboration tool I've ever used, and it's also really good when you're creating music by yourself. It's very easy to use and you can literally get started right away. All you might need to buy is a MIDI controller. I've also recently purchased an XKey which is under $100 USD and it works well; I actually use it most of the time even though I have a larger more traditional MIDI controller. I'd take a look at these two as your least expensive way to get into electronic music creation.
Answer: Computer: If the computer you asked this question on belongs to you, is less than about 2 years old and is not an i3, Pentium or Celeron, it should be fine. Older Athlon X2 or Intel Core 2 Duo/Quad machines are passable, but edging toward obsolete for modern DAWs, especially when you'll be using a lot of VST plugins as with EDM. A Celeron, Pentium or other budget PC marketed to casual users for browsing and e-mail won't have the clocks for the real-time parallel processing a DAW needs.
If you really need a new computer, you should be able to get a laptop from Best Buy, or the guts for a new one from NewEgg or TigerDirect, that will be more than enough for DAW work. Look for a CPU in about the $200 range (that'll be newer i5s, older i7s or AMD FX-8xxx), at least 8GB RAM, at least 1TB HDD (and consider investing in an external HDD and/or cloud storage; audio takes a lot of storage), and if you can still get Windows 7 pre-loaded, IMO that's better than Windows 8/8.1 for audio work (a lot of audio drivers and VSTs aren't stable on Win8 yet). You can get a computer with the specs I just gave for well under a grand, especially if you're swapping out the guts of an existing mid-tower (if all you're doing is replacing CPU/RAM/Mobo and adding some HDDs, the tab can be less than $500 for a screaming rig).
The computer above will be all you'll need for quite some time running a DAW, and won't be half-bad for gaming and other processor-intensive uses either. And this is if your current computer will just not cut it; if your computer was marketed as a gaming rig anywhere in the last 4-5 years it should be fine for DAW work. You don't need a MBP, nor a $3000 gaming rig.
DAW: Reaper is free to use indefinitely (the trial is ostensibly 60 days, but they don't lock down or neuter the software after that), and when your conscience (or the nag screen) gets the best of you it's $60 for a personal non-commercial license. It's a very good DAW too, with a very lightweight framework leaving more "sandbox" for the plugins, and a hardcore and fast-growing following in small project studios where it's compared favorably to the popular expensive monoliths like Pro Tools, Logic, Cubase, SONAR and Ableton.
Keyboard: What you'll want for DAW use is a "MIDI Controller" keyboard, as opposed to a "Digital Piano", "Stage Piano/Keyboard" or "MIDI Workstation", all of which have features you don't need or don't do things you'll want. MIDI controllers are available anywhere between $100 and $1000, with really good ones in the $250-$500 range if you don't mind a "synth-action" keybed and less than 88 keys. Check out the Novation Impulse, Akai MPK and M-Audio Axiom lines, which all come in frame sizes ranging from 25 to 61 keys, and also include pad triggers and assignable fader, knob and button controls for mouseless control of basic DAW and plugin functions. If you're a piano player and really want that same feel, there are a few controllers with 88-key "hammer-weighted" keybeds like the M-Audio Oxygen 88, but those cost more especially if you want the pads and other assignable controls. You might look in the used market for a Roland Fantom or Yamaha Motif workstation; these retail for between $1000-$4000, but sell used for half that or less, and would be the last keyboard you'd ever need.
VSTs: Reaper comes with the basics for free: compression/limiting, gating/ducking, EQ, reverb, even pitch correction and a synthesizer and sampler. If you want more, there are a bunch of quality freeware VSTs available online.
Sample Libraries/Virtual Instruments: Usually a notable subclass of VSTs, sample libraries controlled by a MIDI-compliant software engine basically give you a computer-based version of what you'd have in a $1600 workstation (and sometimes much more). Kontakt is dominant, nearly ubiquitous; the full version is a bit spendy at $400, but there's a free version, Kontakt Player, that's bundled with third-party patch libraries (most still pay-to-play unfortunately), and also available separately with a few freeware libraries like NI's own Factory Selection. The big downside of staying free is that you can only use sample libraries not specifically marked for Kontakt Player for 15 minutes at a time; for unlimited use of most of the freeware NKI libraries available for Kontakt, you need the full version.
All is not lost, however. To get your feet wet, I recommend staying old-school by using SoundFonts. These are an older library standard developed by Creative Labs for their SoundBlaster cards, and picked up by Yamaha, Roland and others as an easy way to load and transfer samples between equipment. There is a free VST available from Cubase, SFZ, which plays any of the massive number of SoundFont files available for free online. Sinfonia is a common go-to SoundFont for orchestral-type instruments, and Chorium is also a popular one still used today.
There are also freeware synth VSTs. Synth1 and Crystal are two tools practically every home recordist has in their arsenal; Synth1 is an excellent virtual analog synth, has a lot of excellent sounds in it reminiscent of the trance heyday of the late '90s, early '00s. Crystal is a sampler, with free patch libraries available from a number of sources.
Existing Music: YouTube is free.
Other things you may need that you didn't list:
Microphone: Vocals aren't a requisite for EDM, but they're common. You'll also probably want some sort of recording setup for sampling. An AT2020 is available for around $50 and it's an excellent place to start for a studio mic. You can also get the Sterling SP30 and SP50 as a set for $60, though this is starting to venture into a really hit-or-miss pricepoint. The MXL990 and 991 are also a common entry-level studio condenser set, of much better quality, for about $120.
Audio Interface: If you want to use pro-audio microphones, you'll need an audio interface to get the signals into the computer; the average consumer-grade sound card mic and line inputs don't support these types of sources. There are plenty of great products in the $150-$200 range. I have the Focusrite Scarlett 2i4 and like its simplicity (USB-powered, no control software) and flexibility (2 inputs for XLR or 1/4" TS/TRS, 4 outputs either TRS or RCA, DIN I/O for older MIDI devices, and very flexible "direct monitoring" capabilities). Its microphone gain leaves a bit to be desired for low-output dynamic microphones, and high-Z inputs peak easily, but it's fine for condenser mics and line-in sources. Other good choices include the Scarlett 2i2, Presonus AudioBox, Steinberg UR22, and Mackie Onyx Blackjack.
Good listening equipment: When you're building your mixes, you'll need equipment that faithfully reproduces what you're listening to, so when you apply an EQ or set the relative level of any two channels, the resulting sound you hear is not colored by a natural response curve of "hi-fi" systems (even expensive consumer brands like Beats Audio or JBL) that emphasize bass and high mids, or cheap computer speakers with totally unpredictable response ranges. Studio reference headphones and near-field monitors are designed for flat, extended response curves, so you hear everything in your mix, and the amount of each frequency is accurate. This generally isn't as pleasing to the ear as a hi-fi system's response, but that's for a good reason; if it sounds good in studio monitors, it'll sound good through almost anything else.
Reference-quality gear is spendy; monitoring headphones typically run from $100 to $300 (I got my ATH-M45s on sale for half price at $50) and "near-field monitors" with flat response curves and good bass response, designed to be placed in an equilateral triangle with your head about 1m on a side, start at about $250 per speaker. For now, you can use a hi-fi shelf system to listen at your workstation, and you should also take the mix around to any other audio system you have access to, from your iPod and cheapie ear buds to your home theater, your car stereo, any pro audio system you or a friend might have for DJing, etc etc. If it sounds good through all of these, then it's good to go.


Como trabalhar com índice no thymeleaf?

Question: Através de um tutorial conseguir criar uma inclusão de produtos utilizando o seguinte código html
<c:forEach items="${tipos}" var="tipoPreco" varStatus="status">

                ${tipoPreco}
                <input type="text" name="preco[${status.index}].valor">
                <input type="hidden" name="preco[${status.index}].tipo" value="${tipoPreco}">

        </c:forEach>

Que fica da seguinte forma;

ela está vinculado a três entidades
De Produto;
@ElementCollection
private List<Preco> preco;

De Preco
private TipoPreco tipo;

E de TipoPreco que é uma enum
EBOOS,IMPRESSO,COMBO;

Estou tendo uma enorme dificuldade em realizar os códigos que fiz em JSP para fazer em thymeleaf.
Eu sei que existe algo em relação com o th:each
<option th:each="status : ${todosStatusTitulo}" th:value="${status}" th:text="${status.descricao}"></option>

eu fiz uma tentativas, mas foram em vão, preciso de sugestões de como poderia colocar o código do thymeleaf para ter o mesmo resultado a que eu fiz com JSP.
================================ATUALIZAÇÃO============================
Fiz a seguinte tentativa seguindo a sugestão proposta;

    <input type="text"  th:name="preco[${status.index}].valor"/>
    <input type="hidden" th:name="preco[${status.index}].tipo" th:value="${tipoPreco}"/>

Gostaria de deixar claro como está o método que carrega o formulário;
@RequestMapping("form")
    public ModelAndView form(){
        ModelAndView modelAndView = new ModelAndView("produtos/form");
        modelAndView.addObject("tipos", TipoPreco.values());

        return modelAndView;
    }

E diante da alteração feita gerou esse erro na pagina;
This application has no explicit mapping for /error, so you are seeing this as a fallback.

Sun Apr 10 06:48:06 BRT 2016
There was an unexpected error (type=Internal Server Error, status=500).
Could not parse as expression: "preco[${status.index}].valor" (produtos/form:30)

E essa foi a mensagem de erro no consoles;
org.thymeleaf.exceptions.TemplateProcessingException: Could not parse as expression: "preco[${status.index}].valor" (produtos/form:30)
    at org.thymeleaf.standard.expression.StandardExpressionParser.parseExpression(StandardExpressionParser.java:238) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.standard.expression.StandardExpressionParser.parseExpression(StandardExpressionParser.java:79) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.standard.expression.StandardExpressionParser.parseExpression(StandardExpressionParser.java:40) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.standard.processor.attr.AbstractStandardSingleAttributeModifierAttrProcessor.getTargetAttributeValue(AbstractStandardSingleAttributeModifierAttrProcessor.java:65) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.processor.attr.AbstractSingleAttributeModifierAttrProcessor.getModifiedAttributeValues(AbstractSingleAttributeModifierAttrProcessor.java:59) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.processor.attr.AbstractAttributeModifierAttrProcessor.processAttribute(AbstractAttributeModifierAttrProcessor.java:62) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.processor.attr.AbstractAttrProcessor.doProcess(AbstractAttrProcessor.java:87) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.processor.AbstractProcessor.process(AbstractProcessor.java:212) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.dom.Node.applyNextProcessor(Node.java:1017) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]
    at org.thymeleaf.dom.Node.processNode(Node.java:972) ~[thymeleaf-2.1.4.RELEASE.jar:2.1.4.RELEASE]

Ele está dizendo com essa mensagem de erro que ele não reconhece esse trecho do código como uma expressão regular do  thymeleaf.
<input type="text"  th:name="preco[${status.index}].valor"/>

Talvés a abordagem do thymeleaf para tratar index seja diferente do JSTL do JSP.
======================MAIS UMA TENTATIVA============================
Encontrei esse link
http://www.thymeleaf.org/doc/tutorials/2.1/thymeleafspring.html#the-springstandard-dialect
e encontrei esse exemplo;

                1
                Thymus Thymi
                12

e tentei fazer assim;

        1
        Thymus Thymi
        12

E deu esse erro;
org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 6): Property or field 'rows' cannot be found on object of type 'br.com.casadocodigo.model.TipoPreco[]' - maybe not public?
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:224) ~[spring-expression-4.2.5.RELEASE.jar:4.2.5.RELEASE]
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:94) ~[spring-expression-4.2.5.RELEASE.jar:4.2.5.RELEASE]
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.access$000(PropertyOrFieldReference.java:46) ~[spring-expression-4.2.5.RELEASE.jar:4.2.5.RELEASE]
    at org.springframework.expression.spel.ast.PropertyOrFieldReference$AccessorLValue.getValue(PropertyOrFieldReference.java:374) ~[spring-expression-4.2.5.RELEASE.jar:4.2.5.RELEASE]

============================SEGUNDA ATUALIZAÇÃO===============
Eu coloquei meu código dessa forma;

                    <input th:text="${tipoPreco}"/>

E ao gerar os input´s eu tentei cadastrar esse registro e não tive sucesso em inserir registros nas base de dados no ebook, no impresso e combo, isso porque ele somente apresenta o nome dos valores que estão na classe enum, tenho que ter uma maneira de acessar a classe preco para poder inserir os valor
veja como está ficando no navegador

Answer: Quando se usa o th:each o Thymeleaf oferece um mecanismo para você controlar a iteração, a variável de status. Ela é declarada após a variável de iteração.
A variável de status contém algumas informações tais como: index, count, size, current e outros que você pode olhar aqui.
No seu código fonte que está no GitHub, você está tentando iterar um Array de Enum. Logo os métodos ou atributos não estão disponíveis por isso acontece o erro:
property or field 'rows' cannot be found on object of type 'br.com.casadocodigo.model.TipoPreco[]' - maybe not public?

Troquei o seu código por este:

E resultou na saída:

Nessa linha
 declarei primeiro a váriavel de iteração e depois a variável de status para manipulação de iteração.
Recomendo você ler a documentação do Thymeleaf. Em PDF tem pouco mais de 85 páginas.
Comment: Recomendo você fechar essa pergunta, marcando alguma resposta como aceita, pois ela se trata do problema com índice,  e abrir uma pergunta sobre Bind de formulários com Spring MVC.
Comment: eu atualizei minha postagem.
Comment: Ai já entra a parte de cadastro e não trabalhar com th:each. Vai ser preciso analisar com mais calma todo seu código para ver o problema. E qual foi o erro que ocorreu quando você tentou cadastrar?
Comment: ele somente  cadastrou titulo, descrição e paginas na base de dados.
Comment: O Spring MVC não está conseguindo fazer o bind, da forma que você deseja, do form para sua entidade de negócio.
Comment: é isso mesmo que está acontecendo.
Answer: Faltou apenas concatenar o texto estático do th:name com o valor dinâmico vindo do status. O código correto seria esse:

    <input type="text" th:name="'preco['+${status.index}+'].valor'">
    <input type="hidden" th:name="'preco['+${status.index}+'].tipo'" th:value="${tipoPreco}">

Answer:

    <input type="text" th:name="'preco['${status.index}'].valor'">
    <input type="hidden" th:name="'preco['${status.index}'].tipo'" th:value="${tipoPreco}">

Comment: acabei de atualizar a postagem, poderia dá uma olhada, por favor!
Comment: fiz uma segunda atualização.
Comment: Tente isso `modelAndView.addObject("tipos", new ArrayList(TipoPreco.values()));`
Comment: somente basta modificar o controller e deixa do jeito que você sugeriu a cima na sua ultima postagem de sugestão?
Comment: `@RequestMapping("form")
    public ModelAndView form(){
        ModelAndView modelAndView = new ModelAndView("produtos/form");
        modelAndView.addObject("tipos", new ArrayList(TipoPreco.values()));

        return modelAndView;
    }`
Comment: Ele não permite tal modificação no controller porque na tabela produtos já existe uma lista com o atributo preco como lista.
Comment: O meu projeto não está em fase de conclusão, na verdade eu criei no dia que fiz essa postagem. quer que eu poste meu projeto no GitHUb para você dá uma olhada?
Comment: Sim. Assim vai ficar mais fácil te ajudar
Comment: meu projeto se encontra aqui >>>   https://github.com/wladimirbandeira/casadocodigo/tree/master/casadocodigo
Comment: se tiver dificuldade para importar o projeto, me informe, por favor
Comment: algum resultado?

Como trabalhar com índice no thymeleaf?


Tennis: When playing doubles, is it not allowed for both players to be at the baseline?

Question: What are the rules when playing doubles? If the serving player (or the one receiving the serve) is naturally positioned at the baseline, should his/her partner be at the net (or more generally, at "the front")? If so, is it a requirement or simply a "best practice"?
I'm asking because I prefer to always stay at the baseline, but I'm often pestered by my partners who insist that I go to the net when they are serving (or receiving the serve).
PS: I'm not asking whether my choice is "strategically" the best, but rather whether it is a allowed by the official rules.
Answer: In general, only the server prior to the serve is constrained on where to stand.  Other players may stand anywhere that does not interfere with their opponent.
Yes, you don't want to distract your partner, and strategically you don't want to abandon the net.  But there is no rule prohibition on standing back when your partner serves.
From the USTA Friend at Court Rules:

Hindrance

...
Case 5: In doubles, where are the server's partner and receiver's partner allowed
to stand?
Decision: The server's partner and the receiver's partner may take any position on
their own side of the net, inside or outside the court. However, if a player is creating a
hindrance to the opponent(s), the hindrance rule should be used.


How to use Tomcat 8.5.x and TomEE 7.x with Eclipse?

Question: I need to setup a Tomcat 8.5.x server version  in Eclipse. When I try to create a server using Eclipse GUI, I get in Eclipse Luna as choice the latest version "Apache Tomcat v8.0". When I select it and I browse to my Tomcat 8.5.x server, I get this blocking error message:

The Apache Tomcat installation at this directory is version 8.5.0.  A
  Tomcat 8.0 installation is expected.

The same error appears when trying TomEE 7.x, which is internally based on Tomcat 8.5.
In Eclipse Neon the latest version as choice is "Apache Tomcat v9.0" and it gives the same error. There isn't even a "Apache Tomcat v8.5" option.
Is there a way to use Tomcat 8.5 and TomEE 7.x in Eclipse? How?
Answer: You have to patch catalina.jar, as this is version number the WTP adapter looks at. It's a quite useless check, and the adapter should allow you to start the server anyway, but nobody has though of that yet.
For years and with every version of Tomcat this is always a problem.
To patch you can do the following:

cd [tomcat or tomee home]/lib
mkdir catalina
cd catalina/
unzip ../catalina.jar
vim org/apache/catalina/util/ServerInfo.properties

Make sure it looks like the following (the version numbers all need to start with 8.0):
server.info=Apache Tomcat/8.0.0
server.number=8.0.0
server.built=May 11 2016 21:49:07 UTC

Then:

jar uf ../catalina.jar org/apache/catalina/util/ServerInfo.properties
cd ..
rm -rf catalina
Comment: +1. For those using Windows OS, you can make the required changes right in the JAR file. Just open the _catalina.jar_ with your file archive utility (WinRar, WinZip, 7zip etc), navigate to the _org\apache\catalina\util\ServerInfo.properties_ file, make changes and save them (confirm that you want to update the archive).
Comment: **UPDATE** And you ONLY need to change the version in the `server.info=` line (like _server.info=Apache Tomcat/8.0.0_), and **server.number** can be left as it is. For instance I have left it untouched like this: `server.number=220.127.116.11`. Works without a problem.
Comment: -1: modifying the server to claim to be a different version is a bit of hack, when clearly the appropriate thing to do is to modify eclipse to accept the actual version *which has been done and is described in other answers*.
Comment: In Tomcat 8.5.3, catalina.jar is in the lib folder.
Comment: After applying this hack, I got "Unknown version of Tomcat was specified." error
Comment: @PeriataBreatta It's a hack for sure. Perhaps you should link to those other answers ;)
Comment: @AtmeshMishra Check the file permissions of catalina.jar.
Comment: @AtmeshMishra I got the same error, and startup.bat refused to run, even when I changed catalina.jar back to the way it was orginally. Ended up having to reinstall tomcat
Comment: Note that in order to do the updates to the ServerInfo.properties file, Tomcat must be stopped or else you may hit some locking issues (i.e. The file is in use by another program) .
Answer: There is a patch for Eclipse:
https://bugs.eclipse.org/bugs/attachment.cgi?id=262418&action=edit
Download this patch and put it to the plugins directory of your Eclipse installation. It will replace the default "org.eclipse.jst.server.tomcat.core_1.1.800.v201602282129.jar".
NOTE
After you add this patch you must choose "Apache Tomcat v9.0" when adding a server runtime environment in the Eclipse (Preferences > Server > Runtime Environments).
I.e. this patch allows you to select either Tomcat version 9.x or Tomcat version 8.5.x when adding Apache Tomcat v.9.0 runtime environment.

More details on can be found on the related bug report page:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=494936
Comment: Did not work for Version: Luna Service Release 2 (4.4.2)
my tomcat adapters are missing after the patch
Comment: Note that this update now appears to be included in recent distributions of Neon, so simply choosing the Tomcat 9.x option might work without needing to update the plugin.
Comment: I'm using the eclipse version Mars.2 (4.5.2) and applied this patch. Now the existing tomcat adapters are all gone. This is crazy patch, I'm rolling it back now.
Answer: For Tomcat 8.5.x users
You've to change the ServerInfo.properties file of Tomcat's /lib/catalina.jar file.
ServerInfo.properties file contains the following code
server.info=Apache Tomcat/8.5.4
server.number=188.8.131.52
server.built=Jul 6 2016 08:43:30 UTC

Just open the ServerInfo.properties file by opening the catalina.jar with winrar from your Tomcat's lib folder
ServerInfo.properties file location in catalina.jar is /org/apache/catalina/util/ServerInfo.properties
Notice : shutdown the Tomcat server(if it's already opened by cmd) before doing these things otherwise your file doesn't change and your winrar shows error.
Then change the following code in ServerInfo.properties
server.info=Apache Tomcat/188.8.131.52.4
server.number=220.127.116.11
server.built=Jul 6 2016 08:43:30 UTC

Restart your eclipse(if opened). Now it'll work...
Comment: Does it work on OS X too? I've just patched my catalina.jar and it doesn't work (nothing has changed after the patch). My Eclipse version is:
Eclipse Java EE IDE for Web Developers.
Version: Mars.2 Release (4.5.2)
Build id: 20160218-0600
Comment: Sure it'll work on OS X too.Initially, I did these changes on my Mac only. After that I planned to use Tomcat on windows. So I just did it and posted as answer here. Give it a try... @piotrdab
Answer: As for now Eclipse Neon service release is available. So if someone is still encounters this trouble, just go to
Help → Check for Updates
and install provided updates related to : org.eclipse.jst
Comment: I confirm that *Eclipse Neon.1 (4.6.1)* add a *Tomcat v8.5 Server*
Comment: I have _Eclipse Java EE IDE for Web Developers_ version _Neon.1 Release (4.6.1)_. Among older Tomcat vestions I also have `v8.0`, `v8.5` and `v9.0` available in _Define a New Server_ window. Maybe Eclipse version you use is different.
Comment: Note that you still need to select *Apache Tomcat v9.0* from the list of server runtimes, which is a little counter-intuitive, but it does work.
Comment: Hmm..  I have 4.6.0 with WST upgraded via the suggested action and still don't have Tomcat 8.5 on the list... but Tomcat 9.0 does work.  Bizarre.
Comment: Eclipse Neon.3 (4.6.3) also has a Tomcat v8.5 Server
Comment: I had the same exact issue, running on ubuntu 18.04, Tomcat v8.5.50 and eclipse v4.11 . and this solution worked for me perfectly.
Answer: This workaround worked for me. I edited the serverInfo.properties file as given below:
server.info=Apache Tomcat/8.0.0
server.number=220.127.116.11
server.built=Oct 6 2016 20:15:31 UTC

Answer: I had similar issues with Eclipse Kepler v3.8
I had tomcat v8.5.37 installed. I couldn't see Apache v8.5 as an option. By skimming through StackOverflow I found Apache v9.0 is available on Eclipse Neon.
Cool thing is you don't have to change your eclipse version.
In your current Eclipse. Download WTP(Web Tools Package) by following the steps:
Step 1: Help >>> Install New Software. Copy this link in the Work with: http://download.eclipse.org/webtools/repository/neon
Step 2: Select JST Server Adapters and JST Server Adapters Extensions from the first package you see. Install those.
Step 3: Windows >>> Preferences >>> Server >>> Runtime Environments >>> Add..
You'll see Apache v9.0 there!
It works!
Comment: Thanks! I had just (re)installed Eclipse 2019-03 on my MacBook, and Tomcat is always the hardest plugin for me to find. I used https://download.eclipse.org/webtools/repository/2019-03/
Answer: Install the latest version of eclipse(). It would have the option to add Tomcat 8.5.
Answer: Go to the preview version of tomcat e.g. : tomcat 8.3 and copy catalina.jar file and
paste into the existing tomcat which you have facing the issue
Answer: Forgive me for invoking old problem. But it is like legendary, always happen for new users. The reason I am here is I want to purpose different answer. Rather simple. Please fo to windows->preference->Runtime Environment->search and select the folder where you download the server. It will automatically detect the server and you are good to go.
Answer: I'm guessing that you are running Eclipse Mars, or an even earlier release. You need to upgrade to Eclipse Neon or later
Answer:
Navigate to /apache-tomcat-8.5.65/lib/org/apache/catalina/util/ServerInfo.properties
Then change
server.info=Apache Tomcat/8.5.65
server.number=184.108.40.206
server.built=Jul 6 2021 00:29:43 UTC

server.info=Apache Tomcat/220.127.116.11.4
server.number=188.8.131.52
server.built=Jul 6 2016 08:43:30 UTC


pyspark remove just consecutive duplicated rows

Question: After some iterations and joins by batch_date, I have a dataframe. Lets focus on r_id0==0 and r_id0==1:
+-----+----------+----------+----------+--------------------+--------------------+
|r_id0|batch_date|      from|     until|                desc|                hash|
+-----+----------+----------+----------+--------------------+--------------------+
|    0|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 1|b10192333e9c61a40...|
|    0|2020-04-30|2020-04-30|9999-12-31|  Esto es un campo 1|b10192333e9c61a40...|
|    1|2020-01-31|2020-01-31|9999-12-31|  Esto es un campo 1|b63b3e8201cd417bb...|
|    1|2020-02-28|2020-02-28|9999-12-31|  Esto es un campo 1|b63b3e8201cd417bb...|
|    1|2020-03-31|2020-03-31|9999-12-31|  Esto es un campo 1|b63b3e8201cd417bb...|
|    2|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 2|b31c138a4b6a96169...|
|    2|2020-02-28|2020-02-28|2020-03-30|  Esto es un campo 4|fa1cfe1eaf9b14f88...|
|    2|2020-03-31|2020-03-31|2020-04-29|  Esto es un campo 3|e4ffd0cd4a6cf1193...|
|    2|2020-04-30|2020-04-30|9999-12-31| Esto es un campo 23|0fdb24b8fcf8603ee...|
|    3|2020-02-28|2020-02-28|2020-03-30|  Esto es un campo 3|a3a6870ca9b42ad06...|
|    3|2020-03-31|2020-03-31|2020-04-29|  Esto es un campo 2|18b24f88271e99618...|
|    4|2020-03-31|2020-03-31|2020-04-29|  Esto es un campo 5|2fe50db0156cfc909...|
|    6|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 6|7c6d329b73d9de59f...|
|    6|2020-04-30|2020-04-30|9999-12-31|Esto es un campo 77|d70c9340f83167e95...|
+-----+----------+----------+----------+--------------------+--------------------+

I do need to remove rows with same hash column, but just the ones regarding consecutive months in the batch_date column. In that case, I take everything from the first time the duplicated appears, and the value of the last until column. Non consecutive duplicates must remain the same.
Example:

r_id0==0 must remain the same. In the batch_date  column there is a gap of 3 months between corresponding entries.
r_id0==1 must be just one row. Its batch_dates months are consecutives.
The result should be like this:

+-----+----------+----------+----------+--------------------+--------------------+
|r_id0|batch_date|      from|     until|                desc|                hash|
+-----+----------+----------+----------+--------------------+--------------------+
|    0|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 42|b10192333e9c61a40...|
|    0|2020-04-30|2020-04-30|9999-12-31|  Esto es un campo 42|b10192333e9c61a40...|
|    1|2020-01-31|2020-01-31|9999-12-31|  Esto es un campo 1|b10192333e9c61a40...|
|    2|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 2|b31c138a4b6a96169...|
etc etc...

I did the function performing what i want for r_id0==1, but it will also modify the r_id0==0 case:
def removing_duplicates2(final_df):
    '''
    Removing duplicates. It takes all from first time they appear and last "until" column.
    '''
    # we create this list in order to avor_id iteration over the column "hash", which will be our groupby column
    iterating_columns= final_df.columns
    iterating_columns.remove("hash")
    exprs =  [F.first(x).alias(x) if x!="until" else F.last(x).alias(x)  for x in iterating_columns]
    return  (
         final_df.groupBy("hash").agg(*exprs)
            .dropDuplicates(["hash"]).select(columns)
         )

This is the result, destroying the r_id0==0:
+-----+----------+----------+----------+--------------------+--------------------+
|r_id0|batch_date|      from|     until|                desc|                hash|
+-----+----------+----------+----------+--------------------+--------------------+
|    0|2020-01-31|2020-01-31|9999-12-31|  Esto es un campo 42|b10192333e9c61a40...|
|    1|2020-01-31|2020-01-31|9999-12-31|  Esto es un campo 1|b63b3e8201cd417bb...|
|    2|2020-01-31|2020-01-31|2020-02-27|  Esto es un campo 2|b31c138a4b6a96169...|
etc etc

So, I'm quite puzzled. I have to do it with Pyspark and this example will be used for a huge, damn huge table partitioned by batch_date.
I strongly believe every loop I create is a step towards servers exploding and firefighters blaming me (I'm using already one (for iterating over batch_date)).
Apologies for the long description,
Any advice or suggestion is more than welcome.
Thanks!
Comment: What exactly do you mean with _adjacent ones regarding batch_date_? How large can the difference between two batch_dates become so that they are still considered to be adjacent? Would the dates `2020-02-14` and `2020-03-31` be close enough?
Comment: Apologies, I'll update the description. The mentioned dates have consecutive months, so yes, it applies for your example
Answer: The idea is to identify in a first step the groups with only consecutive months using a Window. Then group only the rows found in the first step while leaving all other rows unchanged.
Create some test data:
from datetime import date
data = [[0, date.fromisoformat("2020-01-31"), date.fromisoformat("2020-01-31"), date.fromisoformat("2020-02-27"), "b10192333e9c61a40"],
    [0, date.fromisoformat("2020-04-30"), date.fromisoformat("2020-04-30"), date.fromisoformat("9999-12-31"), "b10192333e9c61a40"],
    [0, date.fromisoformat("2020-05-01"), date.fromisoformat("2020-04-30"), date.fromisoformat("9999-12-31"), "b10192333e9c61a40"],
    [1, date.fromisoformat("2019-12-31"), date.fromisoformat("2020-01-31"), date.fromisoformat("9999-12-31"), "63b3e8201cd417bb"],
    [1, date.fromisoformat("2020-01-31"), date.fromisoformat("2020-01-31"), date.fromisoformat("9999-12-31"), "63b3e8201cd417bb"],
    [1, date.fromisoformat("2020-02-28"), date.fromisoformat("2020-02-28"), date.fromisoformat("9999-12-31"), "63b3e8201cd417bb"],
    [1, date.fromisoformat("2020-03-31"), date.fromisoformat("2020-03-31"), date.fromisoformat("9999-12-31"), "63b3e8201cd417bb"],
    [2, date.fromisoformat("2020-03-31"), date.fromisoformat("2020-03-31"), date.fromisoformat("9999-12-31"), "abcd"]]

df=spark.createDataFrame(data, schema=["r_id0","batch_date","from","until","hash"])

Identify those values of r_id0 that contain only consecutive months:
from pyspark.sql import functions as F

w = Window.partitionBy("r_id0").orderBy("batch_date")
df2=df.withColumn("prev_bd_month", F.month(F.lag("batch_date").over(w)))\
    .withColumn("prev_bd_year", F.year(F.lag("batch_date").over(w))) \
    .withColumn("adj", F.when((F.year("batch_date").eqNullSafe(F.col("prev_bd_year"))
                              & (F.month("batch_date") - F.col("prev_bd_month") == 1))
                              | ((F.year("batch_date") - F.col("prev_bd_year") == 1)
                              & (F.month("batch_date") == 1) & (F.col("prev_bd_month") == 12) )
                              | F.col("prev_bd_year").isNull() ,1).otherwise(None)) \
    .groupBy("r_id0") \
    .agg(F.count("*").alias("count_all"), F.sum("adj").alias("count_adj")) \
    .withColumn("all_adj", F.col("count_all") == F.col("count_adj")) \
    .drop("count_all", "count_adj") \
    .join(df, "r_id0") \
    .cache()

Intermediate result:
+-----+-------+----------+----------+----------+-----------------+
|r_id0|all_adj|batch_date|      from|     until|             hash|
+-----+-------+----------+----------+----------+-----------------+
|    0|  false|2020-01-31|2020-01-31|2020-02-27|b10192333e9c61a40|
|    0|  false|2020-04-30|2020-04-30|9999-12-31|b10192333e9c61a40|
|    0|  false|2020-05-01|2020-04-30|9999-12-31|b10192333e9c61a40|
|    1|   true|2019-12-31|2020-01-31|9999-12-31| 63b3e8201cd417bb|
|    1|   true|2020-01-31|2020-01-31|9999-12-31| 63b3e8201cd417bb|
|    1|   true|2020-02-28|2020-02-28|9999-12-31| 63b3e8201cd417bb|
|    1|   true|2020-03-31|2020-03-31|9999-12-31| 63b3e8201cd417bb|
|    2|   true|2020-03-31|2020-03-31|9999-12-31|             abcd|
+-----+-------+----------+----------+----------+-----------------+

Group rows with all_adj == true and keep all other rows:
df3=df2.filter("all_adj == true") \
    .groupBy("r_id0") \
    .agg(F.min("batch_date").alias("batch_date"),
         F.expr("min_by(from, batch_date)").alias("from"),
         F.max("until").alias("until"),
         F.expr("min_by(hash, batch_date)").alias("hash")) \
    .union(df2.filter("all_adj == false").drop("all_adj"))

Result:
+-----+----------+----------+----------+-----------------+
|r_id0|batch_date|      from|     until|             hash|
+-----+----------+----------+----------+-----------------+
|    1|2019-12-31|2020-01-31|9999-12-31| 63b3e8201cd417bb|
|    2|2020-03-31|2020-03-31|9999-12-31|             abcd|
|    0|2020-01-31|2020-01-31|2020-02-27|b10192333e9c61a40|
|    0|2020-04-30|2020-04-30|9999-12-31|b10192333e9c61a40|
|    0|2020-05-01|2020-04-30|9999-12-31|b10192333e9c61a40|
+-----+----------+----------+----------+-----------------+


assumption of well-ordering principle

Question: An abstract algebra book says "the basic assumption here is 'well-ordering principle' : any nonempty set of nonnegative integers has a smallest member." and then using it, the book proves mathematical induction. Can we assume well-ordering principle because it is a theorem?
Comment: You haven't identified the book and you haven't said what its "well-ordering principle" states. If it states that every set can be well-ordered, then this is equivalent to the axiom of choice (AC). However, the principle of proof by mathematical induction over the natural numbers does not require AC. Please give some more details about what your book actually says.
Comment: If you're working with the positive integers, user, you can take well-ordering as an axiom and prove induction, or you can take induction as an axiom and prove well-ordering.
Comment: But see the very interesting paper, Öhman, L., Are Induction and Well-Ordering Equivalent?. Math Intelligencer 41, 33–40 (2019). https://doi.org/10.1007/s00283-019-09898-4 available at https://link.springer.com/article/10.1007/s00283-019-09898-4 where it is noted that if you take the Peano postulates and replace induction with well-ordering then you can't prove induction, as the ordinal $\omega+\omega$ is a model for the modified Peano postulates but $\omega$ has no predecessor so induction doesn't work.
Comment: There's more than one way to define the natural numbers. You can define them as sets in, say, Zermelo-Frankel set theory; you can define them as an ordered integral domain with induction or (equivalently, in this case) well-ordering; you can define them by the Peano axioms. Everything (in algebra) *can* be derived from set theory, but that doesn't imply that set theory is the only, or even the best, way to define any particular algebraic concept.
Comment: @GerryMyerson Thank you very much for the reference! I thought abstract algebra is based on set theory and everything in abstract algebra has to be derivable from set theory. A lot of things are mixed in my head so I am confused.
Answer: Yes, the well-ordering property of the integers follows from the construction of $\Bbb N$ (or $\omega$) in standard set theory (minimal inductive set, e.g.) It's a theorem of ZF.
Comment: Indeed, galactically less than ZF is needed.

assumption of well-ordering principle


How does garbage collection collect self-referential objects?

Question: If an object is not referenced by any other, then it is subject to be collected by the .NET CLR garbage collector.
However, if objA references objB, objB references objC, and objC references back to objA, how does the garbage collector figure out that they (as a whole) can be collected?
Comment: .NET uses a [mark and sweep algorithm (http://stackoverflow.com/questions/2344240/what-is-relation-between-gc-finalize-and-dispose).
Comment: oh yeah, search for roots, i should think about that!
Comment: I don't know for sure but I would assume it uses some kind of tree type thing and anything that isn't connected to the main tree can never be accessed from the main tree (by which I am talking the code that is currently somewhere in the stack). I don't know the specifics but graph theory I'm sure will solve the problem. :)
Answer: The CLR uses a technique known as mark-and-sweep.
As part of this technique, every object can be thought of as initially marked for collection. Then, the CLR goes through each accessible object, starting with your globals (static fields, etc.) as the roots, and clears the mark on each walkable object. It then sweeps the remaining marked objects.
Keep in mind that this "marking" is conceptual; in reality, the objects are most likely added to a collection-set.
In the case of looping self-referenced objects, no reference to the objects will be found from the application, and so the algorithm will never reach those objects to "unmark" them.
Answer: GC has a list of all created objects. During garbarge process it starts from global roots (like static fields) and walk through each referenced object. Each object from the list of all which has not been hit can be destroyed.
If there is no way to hit objA, objB or objC, all these objects will be collected
Comment: @AZ: Those are the heap generations, and it's not a list of the objects, it's the actual objects.
Comment: @Guffa: How does it find the objects that need collecting then if it doesn't have a list of all the possibilities? The mark and sweep descriptions certainly imply that it does...
Comment: Chris: The GC doesn't actually "mark" objects; it is a conceptual construct. In reality, it is objects that the GC **can** access which get marked in the negative. In other words, think of an imaginary property that exists on all objects, called `mark`. `mark` is true by default, so the GC only needs to mark objects it **can** access false.
Comment: -1: So Guffa is correct; the GC does not have a list of all created objects.
Comment: Of course it has list of all created objects. Several lists in fact. They are called generations. One object (pointer) gets put and moved from one list to another during it's lifetime
Comment: ok. if the GC does not have any lists of allocated pointers then something does not match. It runs by marking known objects and decides to keep them. Then on collection it has to dump all "unmarked" objects. If it has no knowledge of them it just has to consider that memory area as usable again as a blob. What about finalizers?. How can it decide what to run and where? So no i don't agree with your assumption.
Comment: @Guffa "No, the GC doesn't have a list of all created objects". Mark sweep requires a list of all allocated objects. The sweep phase reclaims all allocated objects that have not been marked.
Comment: @Novakov I'm sorry to see your correct answer being plagued with completely wrong comments.
Comment: @DavidPfeffer "The GC doesn't actually "mark" objects; it is a conceptual construct". On the basis of my own performance measurements, production mark-sweep and mark-region GCs are unlikely to do anything except maintain a mark bit for each allocated object. You can accumulate a set of marked objects and not have any overhead for unmarked objects but that is much slower and does not save memory.
Comment: @DavidPfeffer "So Guffa is correct; the GC does not have a list of all created objects". No, Guffa is wrong in the context of mark-sweep. The GC needs pointers to unreachable objects in order to reclaim them.
Comment: @AZ. "Of course it has list of all created objects. Several lists in fact. They are called generations". Mark sweep requires a list of all objects but earlier generations are not mark-sweep. They use copying and only require the ability to evacuate survivors, so they only need pointers to reachable objects and not all allocated objects. This lets them sweep all unreachable objects in constant time.
Comment: @JonHarrop: "Mark sweep requires a list of all allocated objects." Yes, of course, but that list is not kept. It's created when needed.
Comment: @JonHarrop: "The GC needs pointers to unreachable objects in order to reclaim them." No, it doesn't. The garbage collector doesn't need the objects, it only wants the memory area where the objects were, and that is simply all the used memory in the heap without any active references to it. If a certain memory area contained one or a thousand objects is irrelevant, the garbage collector doesn't care about the objects once they are unused.
Comment: @Guffa "Yes, of course, but that list is not kept. It's created when needed". It is not possible to recreate the list of all allocated values from the set of reachable values.
Comment: @Guffa "The garbage collector doesn't need the objects". I didn't say the garbage collector needs the objects. My point was that some GCs do keep lists of all currently-allocated values. My own GCs are examples of this. See, for example http://www.ffconsultancy.com/ocaml/hlvm/
Comment: @JonHarrop: You said "The GC needs pointers to unreachable objects in order to reclaim them.". It doesn't.
Comment: @JonHarrop: Some kinds of GC, especially relocating ones, are simply indifferent to objects that were once allocated but don't have any live references.  Instead, they operate on the simple principle that if everything of value that had been in a region of memory has been moved elsewhere, that region of memory can be regarded as containing nothing of value.  If usage patterns fit well with a generational model, then for many objects that contain nothing but PODS and references to other GC objects, allocation will have been nothing more than a pointer add and compare,...
Comment: ...and destruction will require no action whatsoever.  Objects which survive will have to be copied once for each generation, and then once for every "full" GC cycle, but full GC cycles will be relatively rare.  I've been toying with the idea of doing a C++ string-and-array class for small-RAM embedded systems.  I suspect RAM overhead could be reduced far below that of the normal C++ string type.
Comment: @supercat: Other GC algorithms work differently but mark-sweep (as described by John McCarthy in 1958) has pointers to both reachable and unreachable objects. See http://www-formal.stanford.edu/jmc/recursive.pdf
Comment: @JonHarrop: The term "mark-and-sweep" is applied (perhaps not totally correctly) to many kinds of collectors, including some whose sweeping operation totally ignores everything that unreferenced memory had contained.  Such a GC design can reduce memory overhead compared with those that try to keep track of everything.  The string pool in 1980s Microsoft BASIC interpreters used three byte string references (pointer plus length), and zero overhead in the pool itself; it put the length in the reference rather than the target to allow string variables to point into program code.
Comment: @JonHarrop: The GC algorithm MS implemented was pretty terrible, but having each string in the pool preceded by a byte which was known not to be a zero, and have an allocated length of at least three bytes, would have allowed an efficient garbage-collection algorithm, with essentially no change in memory overhead.  I wonder when the first GC algorithms appeared that simply ignored garbage, and if decent ones came before or after bad ones?
Comment: @supercat: "including some whose sweeping operation totally ignores everything that unreferenced memory had contained". I wrote a GC that does that and the interesting thing is, for a significant amount of work to implement that optimisation, you get essentially no improvement whatsoever.
Comment: @JonHarrop: The amount of work required to "ignore" garbage depends I think on how many disjoint free areas will be ready for use after collection.  In relocating collectors that don't support pinning, the number will be *one*.  Some of the more memory-thrifty relocating GC algorithms that do support pinning are time O(N^2) or even O(N^3) on the number of *pinned* objects, but if in expected usage scenarios that N would seldom exceed four or five that's not a problem.  Of course, if *all* objects are pinned (non-relocatable) then such algorithms don't work so well.
Comment: @supercat: "In relocating collectors that don't support pinning, the number will be one". That's only half of the equation. You did a lot of work (copying) to make that number one. That work, in an of itself in the context of mark-sweep, outweighs the benefit you get from being able to deallocate contiguous blocks of memory in one go. FWIW, I regard mark-sweep as inherently non-moving.
Comment: @JonHarrop: The .NET and Java collectors are often described as mark/sweep, but for most objects in .NET which are less than 85K will get copied around by the GC.  The cost is minimized by dividing memory into "generations".  New allocations go into a "generation 0" heap, which is about a meg.  When that gets full, items in there to which references still exist get copied to the gen1 heap; older objects stay where they are.  When the gen1 heap gets full, objects get copied from there to the gen2 heap, while older (gen2) objects stay where they are.  Only when the gen2 heap gets full...
Comment: ...is it necessary for the system to worry about gen2 objects.  The fraction of total objects in the gen0 heap is sufficiently small, and the occurrence of gen2 collections is sufficiently rare, that overall performance ends up being quite good.  Even if one tracks gaps in gen2, and tries to have gen1 objects fill those gaps, only a small fraction of the objects which are created will make it to gen2; those that disappear before that won't have to be tracked.


Intuition behind the null distribution of the deviance statistic in survival models

Question: I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.2 Residuals and Goodness-of-Fit. A goodness-of-fit statistic called deviance is defined as
$$
D = 2 \sum_{i=1}^N n_i \sum_{t=1}^k p_{it} \log\left( \frac{p_{it}}{\hat\pi_{it}} \right)
$$
where $\hat\pi_{it}$ is the estimated probability for a person belonging to group $i$ (of size $n_i$, among $N$ groups in total) to experience an event in the time period $t$ (from 1 to $k$), and $p_{it}$ is the corresponding observed proportion of persons in group $i$ that indeed experienced the event in the time period $t$.
A well-fitting model will produce values $\hat\pi_{it}$ that are close the the observed proportions $p_{it}$, though there will be discrepancies due to sampling variability inherent in $p_{it}$. This will yield $\frac{p_{it}}{\hat\pi_{it}}\approx 1$ and thus $\log\left( \frac{p_{it}}{\hat\pi_{it}} \right)\approx 0$ and thus $D\approx 0$. (Correct me if I am wrong.)
On the other hand, there is a later statement on the same page that asymptotically $D\sim\chi^2(N(k-1)-p)$. Now, I know $\chi^2(N(k-1)-p)$ can be obtained as a sum of squares of $N(k-1)-p$ independent N(0,1) random variables or as a sum of squares of a different number of dependent ones. However, I do not get the intuition why $p_{it} \log\left( \frac{p_{it}}{\hat\pi_{it}} \right)$ should behave like a square of N(0,1). Perhaps it should not? More generally, how is the asymptotic $\chi^2(N(k-1)-p)$ distribution obtained? (Intuition is welcome.)
Moreover, it seems to me that $\log\left( \frac{p_{it}}{\hat\pi_{it}} \right)$ and thus $p_{it} \log\left( \frac{p_{it}}{\hat\pi_{it}} \right)$ can be both lesser and greater than zero, and so $D$ could end up being negative. That is incompatible with a $\chi^2(N(k-1)-p)$ distribution. Am I getting this wrong?
Answer: This section of the Tutz and Schmid book assumes that there is no censoring, so this result is just the application of maximum-likelihood analysis to a multinomial distribution of events over the $k$ time periods.
Following Section 1.5.1 of the second edition of Agresti's Categorical Data Analysis, the multinomial log-likelihood function for a model with a set of probabilities $\pi$ given $n_j$ observations in $j$ categories (total $N$ observations) is:
$$ L(\pi)=\sum_j n_j \log(\pi_j)$$
where $\pi_j$ is the modeled probability of category $j$.
The displayed deviance formula follows the standard definition (e.g., Section 4.5 of Agresti): twice the difference in log-likelihood between a saturated model with the maximum likelihood:
$$\sum_j n_j \log(n_j/N)= \sum_j n_j \log(p_j),$$
where $p_j$ is the observed probability of category $j$, and that of the model in question:
$$\sum_j n_j \log(\pi_j).$$
As the data have maximum likelihood under the saturated model, the likelihood under any other model cannot exceed that. So the sum of terms contributing to the displayed deviance formula is necessarily non-negative, even if individual terms might be negative.
The asymptotic chi-square distribution comes from Wilks's theorem. Agresti provides a proof specific to the multinomial likelihood-ratio statistic in Section 14.3.4. This page and its links provide some intuition about Wilks's theorem. For degrees of freedom, I suppose you can think about this as $(k-1)$ independent time periods (no censoring, so the results for $k-1$ periods determine that for the $k^{th}$ period) for $N$ groups, less the $p$ fitted parameters.
In terms of using the deviance residuals in survival analysis (in models that incorporate censored data), Therneau and Grambsch note in Section 4.3:

The deviance residual was designed to improve on the martingale residual for revealing individual outliers, particularly in plotting applications. In practice it has not been as useful as anticipated.

I can't say that I've found them very useful, unlike the scaled Schoenfeld residuals for evaluating proportional hazards assumptions, or martingale residuals for evaluating functional forms.


Close 8080 for incoming requests on Windows 10

Question: The development environment for REST development needs a Web Container. I am using a locally installed Apache Tomcat.
That laptop is connected to a huge intranet and should be protected against unwanted access from outside.
What is a good approach for this protection?
I tried configuring the Win10 firewall rules and filtering, e.g. applying wf.msc, but without success.
Comment: Does [this](https://learn.microsoft.com/en-us/windows-hardware/drivers/network/introduction-to-windows-filtering-platform-callout-drivers) help you? It's something "similar" to *iptables* on *Windows*. You can create a route that forbids all outside traffic but at the same time allows traffic from inside on one single port.
Comment: @FourBars thanks, it helps. citation: "... extend the capabilities ... beyond ... simple filtering". Let me extend my question: my requirement can not be achieved by *simple filtering*?
Comment: What programming language are you using to code this protection? It looks like you are looking for some Windows solution instead of developing your own one. What does it really have to do with programming?
Comment: @Dharman I am using Python as the client language, Java as the language for the REST service.
Comment: It's fairly simple: With Python I am using the *Requests* module, with Java I am using the *Spring-Boot* eco system.
Answer: I found a method by introducing a Tomcat valve added to %CATALINA_HOME%\conf\server.xml:
<code><Valve className="org.apache.catalina.valves.RemoteAddrValve"
    allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1"/>
</code>
This way the locally installed Tomcat can only be accessed locally.
Comment: Notice that the IP6 part of the regex allows for two representations. [Seems to be important for Win10](https://stackoverflow.com/questions/69786709/why-a-blocking-valve-does-not-work-as-expected-on-win10).


Como deletar arquivos e pastas recursivamente de modo seguro com Python?

Question: Galera, estou tentando criar um script para deletar meus arquivo e pastas de modo seguro, assim como faz o shred, srm, etc...mas eu gostaria de fazer isso com script Python. Eu consegui achar uma função na internet que grava valores aleatórios no arquivo antes de deletar ele. Sei que não existe nada irrecuperável, mas como didática gostaria de implementar um script assim.
Tenho essa função:
def secure_delete(file_, steps=3):
    import os
    with open(file_, "ba+", buffering=0) as f:
        data = f.tell()
    f.close()
    with open(file_, "br+", buffering=0) as f:
        for i in range(steps):
            f.seek(0, 0)
            f.write(os.urandom(data))
        f.seek(0)
        for _ in range(data):
            f.write(b'\x00')
    os.remove(file_)

Passando o arquivo como argumento, consigo fazer a ação, mas eu gostaria de fazer isso recursivamente em um diretório para todos os arquivos e não abrindo um arquivo especifico ou passando ele como argumento.
Alguém tem alguma ideia?
Answer: Gravar dados por cima dos dados de um arquivo não é "seguro" - e o motivo é que depende da camada de filesystem (F.S.) do sistema operacional decidir o que fazer quando você abre um arquivo para escrita  -  e se você for ver, nenhum deles, por vários motivos, vai gravar os dados na mesma posição física do disco em que os dados anteriores estavam.
A ideia de que ao abrir um arquivo para leitura e escrita, você pode modificar um único byte do arquivo, fechar, e ler de novo e ter todos os dados originais com aquele 1 byte diferente, é conveniente para programas em alto nível, mas é só uma abstração do sistema operacional.
Na prática, por conta de como o acesso a dados em disco evoluiu históricamente, o sistema só consegue escrever blocos de, no mínimo, 512 bytes - mas mais provavelmente 4Kbytes (4096) de uma vez - o que é feito quando você modifica um único byte é:
 - as camadas mais baixo nível do S.O. leem 4KB do disco (mesmo que o arquivo seja menor, qualquer sujeira que esteja no disco depois do arqiuvo é lida pra memória)
 - a camada de mais alto nível do S.O. alterar o byte desejado, dentro desse bloco
 - os 4KB são colocados de volta no disco, mas não na mesma posição
 - se tudo correu bem até aqui, a camada de F.S. do sistema operacional altera os metadados do arquivo para lerem os 4KB da nova posição, não da antiga. (principalmente em F.S. modernos que tem um mencanismo de journaling - que permite justamente que  a versão antiga do arquivo seja lida, se acontecer qualquer falha antes do final do processo todo).
Qualquer ferramenta que tenha acesso aos dados brutos na partição de disco pode então acessar os dados de arquivos "sobre-escritos". Pode dar um bom trabalho reconstruir esses arquivos a partir dos pedaços achados - e muitas partes podem sim estar fisicamente sobre-escritas, pelo próprio processo de re-escrever os arquivos, mas é mais ao "acaso" do que destruição deliberada.
Se você eestiver num sistema operacional Linux, os dados brutos da partição são acessíveis simplesmente se abrindo os dispositivos especiais em /dev/ como se forem arquivos normais. (Nesse caso, ler e escrever nesses arquivos de dispositivo, sim, faz os bytes serem lidos e escritos nas mesmas posições físicas do disco - justamente por que o código de filesystem usado pelo kernel para acessar os dispositivos físicos como arquivos faz isso)
Então, se você tiver alguma bibliotca especial, que consiga entender as estrutura de dados do filesystem particular que você pretende alterar, é possível, com acsso ao dispositivo "bruto" (raw), sobre-escrever os bytes exatos, de forma irreversível, dos arquivos, como vocẽ deseja.
O trabalho pra fazer isso de forma correta é pelo menos uma ordem de grandeza mais complexo do que só usar a camada de usuário de abrir e gravar arquvos, como você quer fazer. (alguns filesystems são tão complexos, que dezenas dos melhores devs do mundo levaram mais de 10 anos para acessar seus dados diretamente de forma correta numa implementação paralela - veja o histórico dos drivers de NTFS no Linux, por exemplo, ou o ZFS). Mas se for uma FAT, ou FAT 32 - ainda usado em alguns drives USB, é mais ou menos tranquilo refazer o acesso, e pode ser um projeto bem divertido. (Mas em drives USB, não sei não se a camada de low-level, de firmware do dispositivo, não remapeia os blocos - ou seja, para qualquer acesso em software, sim, os dados estão "sobre-escritos" - mas pode ser que fisicamente ainda estejam lá - em discos SSD, com certeza isso acontece)
Acredito que quanto a sobre-escrever os arquivos recursivamente, as outras respostas aqui já dão conta - então não vou acrescentar ainda um outro exemplo de como fazer.
Comment: Nossa obrigado por esse texto cheio de conhecimento. então seria melhor eu ñ usar essa forma de gravar os dados que usei e sim apenas criptografar os arquivo?
Comment: criptografar os arquivos não adianta nada, nesse contexto: os dados criptografados vão ser escritos numa posição diferente do disco, e o conteúdo original vai estar lá pra quem acessar o dispositivo diretamente.
Criptografar direto o HD, sim, é algo que todos os sistemas operacionais hoje permitem, e, sem digitar a senha correta na hora de ligar o computador, não tem como ler.
Comment: Já, para dificultar o acesso - sem ir ao ponto de sobre-escrever os dados na camada física do disco, gravar um arquivo de mesmo nome na mesma pasta, e depois apaga-lo, é o suficiente: com isso alguém só com uma ferramenta de olhar os metadados do filesystem para "desapagar" um arquivo, vai chegar na posição errada.  Mas você também deve  apagar os caches do sistema operacioal, e cuidar para que mecanismos como o "time machinne" do Mac OS não tenham cópias de versões anteriores do arquivo.
Comment: Entendiiii. E no caso de Linux, apagar os caches...tem como?
Answer: Você pode ler os arquivos do diretório corrente usando os.listdir('.') ou indicar o diretório que quer ler.
Outra forma é usando glob.glob('*.dat') onde a resposta seria uma lista de arquivos, neste caso do exemplo, com extensão ".dat".
A entrada da função poderia ser a lista de arquivos e tratava todos com um "for" ou poderia fazer um "for" e usar a função para cada arquivo.
Se quiser ver arquivos de subdiretórios pode considerar usar a função "os.walk".
Answer: Bom, pegando esse exemplo você pode fazer assim:

import os
import shutil
import uuid

def recursive_listing(path):
    files = []

    # r = root, d = directories, f = files
    for r, d, f in os.walk(path):
        for file in f:
            files.append(os.path.join(r, file))
        for dirs in d:
            files.append(os.path.join(r, dirs))

    list_ = [file for file in files]
    return list_

def secure_delete_recursive(path, steps=5):

    objects = recursive_listing(path)

    for obj in objects:
        # Para arquivos (gravando, renomeando e deletando)
        if os.path.isfile(obj):
            try:

                with open(obj, "ba+", buffering=0) as f:
                    data = f.tell()
                f.close()

                with open(obj, "br+", buffering=0) as f:
                    for i in range(steps):
                        f.seek(0, 0)
                        f.write(os.urandom(data))
                    f.seek(0)
                    for _ in range(data):
                        f.write(b'\x00')

                name = str(uuid.uuid4())
                new_file_rename = os.path.join(os.path.split(obj)[0], name)
                os.rename(obj, new_file_rename)
                # Descomente a linha abaixo para deletar os arquivos recursivamente.
                # os.remove(new_file_rename)

            except PermissionError as p:
                print(p)

    for obj in objects:
        # Para diretórios (renomeando e deletando)
        if os.path.isdir(obj):
            try:

                name = str(uuid.uuid4())
                new_file_rename = os.path.join(os.path.split(obj)[0], name)
                os.rename(obj, new_file_rename)
                # Descomente a linha abaixo para deletar as pastas recursivamente.
                # shutil.rmtree(new_file_rename, ignore_errors=False, onerror=None)

            except PermissionError as p:
                print(p)

if __name__ == '__main__':
    secure_delete_recursive('/tmp')

Você também pode usar o biblioteca Cryptography com essa mesma ideia desse exemplo acima, fica até mais seguro do que gravar binários ou, você pode usar as duas formas juntas que reforma mais ainda. Espero que tenha ajudado.
NOTA: Se não tiver permissão nos arquivos e pastas, não vai funcionar para determinados objetos.
Comment: nossa era isso mesmo que queria. obrigado.
Comment: vou ver sim sobre a biblioteca
Comment: Se não for feita uma chamada  `f.flush()` entre o passo de gravar nḿeros aleatórios e o de gravar 0s, é como se o primeiro passo nem existisse - os números aleatórios na maior parte dos casos, nem serão transferidos para o disco.
Comment: Outra coisa: `                    for _ in range(data):
                        f.write(b'\x00')` é ineficiente nota 15, numa escala que vai de 0 até 10: esão sendo feitas duas mudanças de contexto da aplicação Python para cada _byte_ do arquivo - se for um arquivo de 1GB, pode esperar algumas horas.
O correto aí é `f.write(b'\x00' * len(data)) ` ( claro, para arquivos da ordem de GB, melhor escrever uma estratégia que faça isso em blocos, mas são algumas poucas linhas)
Comment: jsbueno sim é verdade. Arquivos grandes a performance do script seria uma pausa imensa. Não percebi isso no script dele.
Comment: entao tenho que tirar o for _ in range(data): f.write(b'\x00'), ele não precisa? tentei colocar o * len(data) pq sao tipos diferentes.
Answer: Tenho um script que exclui todos aquivos das pastas de cache do Windows, creio que isso pode te ajudar.
import os
import win32con, win32api

list = ['C:\Windows\Prefetch','C:\Windows\Temp']

def clear_data(locate):
    for raiz, diretorios, arquivos in os.walk(locate):
        for arquivo in arquivos:
            try:
                print(arquivo)
                win32api.SetFileAttributes(os.path.join(raiz,arquivo), win32con.FILE_ATTRIBUTE_NORMAL)
                os.remove(os.path.join(raiz,arquivo))
            except:
                print(arquivo+' Erro')

temp = os.getenv('temp')
temp = temp.replace('Roaming','\Local\Temp')
list.append(temp)

for i in list:
    clear_data(i)

exit()

Comment: Humm, não tinha pensado nisso. Obrigado :)
Comment: Mas então - @WillianJesusDaSilva - espero que tenha entendido a ideia de nem precisar criptografar - dados aleatorios são ainda melhores, e de que a extensão não faz a menor diferença. Posto isso, a sua lembrança de excluir os arquivos de cache do sistema é bem pertinente - vou deixar um upvote por conta disso.
Comment: Entendi...mas a função que eu fiz será que ela faz a mesma coisa que o srm, ou shred? Tipo, abrir o arquivo e gravar dados binarios nele para não ser recuperado?
Comment: A que apresentei apenas exclui arquivos de cache de algumas pastas como temp e prefetch.
Comment: dentro do try voce pode alterar tudo aquilo para um objeto contendo a sua função e setar no LIST o diretorio que você quer excluir
Comment: Hum, tendi. Mas mesmo q eu consiga fazer isso, eu precisaria saber a forma de deixar o arquivo irrecuperavel e ilegivel, se agora forma funciona. Vou pegar sua dica pra deixr recursivo
Comment: Eu faria o seguinte.
Transformaria o arquivo em um TXT alterando sua extensão.
Em seguida utilizaria uma lib para encryptar o conteudo do txt utilizando uma hash difícil.
salvaria em txt e após isso excluiria o arquivo.
Comment: e voce quer só destruir o arquivo, nao repcisa "encriptar" o arquivo - só sobre-escrever já faz tudo.
Outra coisa: a extenção de um arquivo não faz a menor diferença para libs, ou para programas rodando, é só uma dica (no caso do windows, para o sistema operacional - no caso do Unix, uma dica mais só para os usuários) de qual programa deve abrir aquele arquivo. O conteúdo em bytes de um arquivo lido não muda qualquer extensão que ele tenha.


What will happen at worst if I change the "Number of values" of CCK field

Question: I have a CCK text field and set "Number of values" to a certain number.
Now I want to change it to "Unlimited".
It says
<code>Warning! Changing this setting after data has been created could result in the loss of data!
</code>
What will happen at worst if I change this?
The values of already existing nodes might be really gone as it says?
If yes, there is no way to change this to "Unlimited" if I already have some existing nodes?
Answer: Changing it to unlimited isn't going to cause an issue. It's a standard warning message for any change on that setting for a field.
So really the only time you would be concerned is if you took an already unlimited field and set it to a static value, which could potentially remove values if the delta of that field on a node exceeded the static value.
Edit: But always be sure to take a backup before doing those kinds of changes, and always do this on a development environment.
Comment: Good point about the backup
Comment: Thanks for the advise. I'll take a backup and change the limit.
Answer: I can't think of any reason why you'd suffer any data loss if you're increasing the field's cardinality (I used to do it all the time in D6 and to the best of my recollection nothing ever went wrong).
The problems would start happening if you decreased the cardinality. For example if if it was previously at 3, and you decreased it 2, any nodes that had a 3rd value for that field  would lose that data.


Reference WPF custom control library to a VSIX project

Question: I have a WPF Custom control project and a VSIX project in my solution.
THe WPF holds the UI windows and controls that are required both by the VSIX and another WPF app that can be used separately from the VS extension.
I have added the project as a dependency in  source.extension.vsixmanifest
like so:
<Asset Type="Microsoft.VisualStudio.Assembly" d:Source="Project" d:ProjectName="StringsRefactor.UI" Path="|StringsRefactor.UI|" AssemblyName="|StringsRefactor.UI;AssemblyName|" />

the code compiles but during runtime when i consturct the required object from thr StringsRefactor.UI assembly i get the following error:

Could not load file or assembly 'StringsRefactor.UI, Version=220.127.116.11,
  Culture=neutral, PublikKeyToken=null' or one of its dependencies. A
  strongly-named assembly is required. (Exception from HRESULT:
  0x80131044)

How should i create a strongly-named assembly?
Or should i do something else ?
Comment: Sign your WPF assembly
Comment: Thank you !!! can you place it as an answer ?
Comment: Fun fact: listing something as Assembly in your manifest doesn't do anything. :-)
Answer: Simply sign your WPF assembly, it is required by Visual Studio


How can I get the list of column fast in R for regression?

Question: I have a dataframe with many columns, and I want to do a regression.
Currently, I have to type:
lm (col3 ~ col4 + col5 + col6 ... + col20)

It is not very convenient. Is there a faster way to do that?
Answer: We subset the dataset by selecting the columns that are needed in the regression formula and use . on the RHS of ~ to specify all the columns that are not the dependent variable ('col3').
lm(col3~., subset(df1, select=col3:col20))

data
set.seed(24)
df1 <- as.data.frame(matrix(sample(0:9, 25*40, replace=TRUE), ncol=25))
colnames(df1) <- paste0('col', 1:25)


Kubernetes Dashboard v1.8.3 deployment

Question: I am just trying to deploy kubernetes Dashboard in a namespace called "test".
https://raw.githubusercontent.com/kubernetes/dashboard/v1.8.3/src/deploy/recommended/kubernetes-dashboard.yaml
I just replaced namespace from kube-system to test  from the above yaml file and executed as below.
kubectl apply -f kubernetes-dashboard.yaml -n test

But, it is still trying to do something with namespace kube-system and getting the below error.
Image:

gcr.io/google_containers/kubernetes-dashboard-amd64:v1.8.3

Error:-
2018/05/31 16:56:55 Starting overwatch
2018/05/31 16:56:55 Using in-cluster config to connect to apiserver
2018/05/31 16:56:55 Using service account token for csrf signing
2018/05/31 16:56:55 No request provided. Skipping authorization
2018/05/31 16:56:55 Successful initial request to the apiserver, version: v1.10.2
2018/05/31 16:56:55 Generating JWE encryption key
2018/05/31 16:56:55 New synchronizer has been registered: kubernetes-dashboard-key-holder-kube-system. Starting
2018/05/31 16:56:55 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2018/05/31 16:56:55 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: unexpected object: &Secret{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Data:map[string][]byte{},Type:,StringData:map[string]string{},}
2018/05/31 16:56:57 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2018/05/31 16:56:57 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2018/05/31 16:56:57 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2018/05/31 16:56:59 Storing encryption key in a secret
panic: secrets is forbidden: User "system:serviceaccount:test:dashboard" cannot create secrets in the namespace "kube-system"

goroutine 1 [running]:
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.(*rsaKeyHolder).init(0xc420254e00)
    /home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:131 +0x2d3
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.NewRSAKeyHolder(0x1a7ee00, 0xc42037a5a0, 0xc42037a5a0, 0x127b962)
    /home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:170 +0x83
main.initAuthManager(0x1a7e300, 0xc4201e2240, 0xc42066dc68, 0x1)
    /home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:183 +0x12f
main.main()
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:101 +0x28c

I created Secret, Rolebinding, Serviceaccount, deployment, Service & Ingress in the namesapce "test". Removed namespace from the yaml file and supplied thru -n "test" while creating.
Answer: That happened because you created the ServiceAccount on a different namespace, namely test but as it says, it needs to be deployed in kube-system in order to be able to function.
You can find a nice walkthrough and possibly some clarifications here
However, if you still want to deploy on a different namespace, you would have to add the following role and rolebinding to your cluster:
# ------------------- Dashboard Role & Role Binding ------------------- #

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
rules:
  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create"]
  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["create"]
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["kubernetes-dashboard-key-holder"]
  verbs: ["get", "update", "delete"]
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["kubernetes-dashboard-key-holder"]
  verbs: ["get", "update", "delete"]
  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["kubernetes-dashboard-settings"]
  verbs: ["get", "update"]
  # Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
  resources: ["services"]
  resourceNames: ["heapster"]
  verbs: ["proxy"]
- apiGroups: [""]
  resources: ["services/proxy"]
  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: test

---

I am afraid there is no other way around, you have to allow the service account to create secrets in kube-system namespace.
Comment: Nope, I replaced the namespace to test for service account also.  apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: test

Kubernetes Dashboard v1.8.3 deployment


Binomial distribution and common sense aren't adding up

Question: I've been trying to work through this in my head and can't figure out what I'm missing. Here's the problem:
A 20 sided die is rolled 100 times and we count the number of 20's we roll. Well, it turns we rolled zero 20's. That's odd! But how odd is it? My first instinct is to say the probability of not getting a single 20 in 100 rolls is  $$\frac{19}{20}^{100} = 0.0059$$
That makes sense to me, but I wanted to reframe it in the context of the binomial distribution with n=100 and p=1/20. I set up my z test as: $$Z = \frac{X-np}{\sqrt{np(1-p)}} = \frac{0-100(\frac{1}{20})}{\sqrt{100(\frac{1}{20})(\frac{19}{20})}} = -2.294$$ which equates to a p-value of ~0.011 or 1.1%. That's a big difference between the two methods and I can't spot a mistake with either. Where'd I go wrong?
Comment: The $p$-value is  Til probability, is based on an asymptotic approximation to the distribution of $X$, and is particularly approximate in the tail $X=0$.
Comment: See https://en.wikipedia.org/wiki/Binomial_distribution#Limiting_distributions
Comment: It's not so much the binomial distribution and common sense not matching up, but the binomial distribution and the normal approximation not doing so.
Comment: An alternative approximation would be to use a Poisson distribution to suggest $e^{-100/20}\approx 0.0067$, which is closer even if not good
Answer: The problem is that what you are referring to is not binomial distribution, but normal approximation of it. As you can learn from the Wikipedia article on binomial, the normal approximation does not always work well and it works mostly for approximating the area of the distribution close to the mean, that's not your case.
Binomial distribution has a probability mass function
$$
{n\choose k} p^k (1-p)^{n-k} = 1 \times 1 \times \frac{19}{20}^{100}
$$