Live-Updating Bar Chart

Dynamic histogram gif
http://bl.ocks.org/charlesdguthrie/11356441

小白码农辛苦上岸的一些经验

全局： 👍   97% (297)     2% (9)    👎 2019(7-9月)-EE硕士+1-3年 | Other| 码农类General全职@ 背景：本来是一行代码不会写的，读了几天HTML/CSS的教程，踩了狗屎运找了个小公司实习，写HTML/CSS，然后return offer，到现在一眨眼就是2年零10个月。突发奇想想去做后端，业余时间开始刷题，努力了一阵刚拿到offer。有个同事问我是怎么学的，给他讲了讲，顺便给大家分享下我的小白上岸之路。 1.  https://www.freecodecamp.org/new ... mmies-5e048933b82b/ -baidu 1point3acres     这是一篇很入门的文章，介绍算法面试的流程/ 题型分类 / 典型题。 建议仔细读一遍，然后把里面的题大概都看一遍，有个概念。 2. cc150/cc189（google Cracking-the-Coding-Intervie pdf 下载）， 基本一样，是几乎所有数据结构的讲解，例题和答案。例题可以读一读，不用写，leetcode里都有。 3. 刷Leetcode，每道题点discussion，都有别人上传的代码，有的题有最优解法，没读过答案根本想不出来，这种题不用较真，多读几次答案就行。google 直接搜题号和题目，有很多同胞给的讲解，挑自己喜欢的风格就行，这三个人我用的比较多     https://www.cnblogs.com/grandyang/p/4606334.html  这个人讲的非常细     https://zxi.mytechroad.com/blog/ ... -number-complement/  这个人是视频讲解     https://wdxtub.com/interview/14520594642530.html 这个人分享的覆盖面比较全 4. 然后就是 刷，刷，刷，刷题，刷200-300道就足够应付中型固定题目的公司了。 5. 1point3acres 同胞分享的面试经验/题目，刷题心得及包裹等等，受益颇多     hired.com 是hr主动上门的网站     triplebyte  免费模拟面试（可重复），然后帮忙联系公司 -baidu 1point3acres 6. 什么时候拿到面试了, 开始准备ood，system design 和 SQL 就来得及，题型非常固定，2-3周足够了。     https://www.educative.io/，付费的，别心疼钱，省去了很多搜索资料的时间     http://blog.gainlo.co/index.php/2016/10/22/design-youtube-part/ ， system design 补充资料，免费的，也不错     SQL 就做leetcode上的就行，不多，主要是熟练，要多写，面试是设计table（随便找一遍文章读一下，难度基本===人有几个鼻子几个眼睛），加白板写query，99%就是join，group by，count，order by， limit 这几个，换汤不换药。 7. 算法读懂了就可以投投简历，很多公司第一步是算法在线测试，就算不想去，就当作模拟题练手。一亩三分地上，只要有人分享面经的公司，就说明在招人，如果有朋友在公司里就找内推， 不然就直接到公司官网投了，就有概率得到测试。 8. 我的timeline，两个半月刷题260道，1.5遍。一个月从零学习的OOD, SYSTEM DESIGN, SQL. 海投了30-50家公司拿到面试3-5个， 内推10家公司拿到面试5个， onsite interview 2次， 一个offer，一个待定， 赶巧拿offer的公司比较想去，package也几乎是现在double，打算签了。 应该是第一次在求职栏发帖，很多规矩不太懂，如果有违规或冒犯之处，请见谅。 人生路很长，祝人人可以先上岸，然后找到自己梦想的靶心。 最不想提，但是还是提了，求米 https://www.1point3acres.com/bbs/thread-563736-1-1.html

DFS + Memoization (Good) vs DFS + Memorization (wrong)

DFS + Memorization


memorization 美 [,mɛmərɪ'zeʃən]  n. 记住；暗记
Memoization Pattern 使用备忘录模式
Memoization is an approach to avoid work repetition by caching previous calculations for later reuse, which makes memoization a useful technique for recursive algorithms. 
制表，通过缓存先前计算结果为后续计算所重复使用，避免了重复工作。这使得制表成为递归算法中有用的技术。

If the terminal condition is correct, then the algorithm contains too much recursion to safely be run in the browser and should be changed to use iteration, memoization, or both. 

如果终止条件是正确的，那么算法包含了太多层递归，为了能够安全地在浏览器中运行，应当改用迭代，制表，或两者兼而有之。

! "bang", "exclamation point"
@ "at", and rarely, "strudel"
# "crunch", "hash", "pound", and rarely, "octothorpe"
^ "circumflex", "hat", "chapeau"
& "ampersand", "and"
* "splat", "star", "asterisk", "times" (as in multiplication)
_ "underscore"
- "hyphen", "dash", "minus sign"
. "dot", "period"
, "comma"
: "colon"
; "semi-colon"
/ "slash"
\ "backslash"
~ "twiddle", also "squiggle", or more correctly, "tilde"
' "tick", "quote", "apostrophe"
" "double-quote"
` "backtick", "backquote"
< "less-than", "left angle bracket"
> "greater-than", "right angle bracket"

https://www.1point3acres.com/bbs/thread-308790-1-1.html

My favorite quotes from movie Aladdin

source:

https://www.moviequotesandmore.com/aladdin-best-movie-quotes/


You got things to do. Places to go. People to see. Futures to make.

A diamond in the rough.  像未经雕刻的原钻

The boss looks ill-tempered, but he is in fact a diamond in the rough.


The candidate pools were so large it's hard to find a diamond in the rough.

Aladdin: We get by. Every day, I just think things will be different, but it never seems to change. Just sometimes, I feel like I’m…
Jasmine: Trapped. Like you can’t escape what you were born into?
Aladdin: Yes.

Jasmine: You remember my mother used to say, “We would only ever be as happy as our least happy subject.” If she saw what I saw today, she would be heart broken.
Dalia: She would also want you to be safe. And clean. I’ll draw the bath.

Steal an apple, and you’re a thief. Steal a kingdom, and you’re a statesman.

Only weak men stop there. You’re either the most powerful man in the room, or you’re nothing. You, you stumbled upon an opportunity. I can make you rich. Rich enough to impress a princess. But nothing comes for free.

Aladdin: What would I have to do?
Jafar: There’s a cave nearby, and in it, a simple oil lamp. Retrieve it for me, and I will make you wealthy enough to impress a princess. You’re nothing to her. But you could be. Your life begins now, Aladdin.


Jafar: [to Aladdin] The Cave of Wonders. When you enter, you will see more riches than you ever dreamed of. Gold, diamonds, and the lamp. Bring it to me and I will make you rich and free. But take no other treasure, no matter how sorely you are tempted. And you will be tempted.
[as Aladdin approaches the cave, it opens up]
Cave of Wonders: Only one may enter here. One whose worth lies far within. A diamond in the rough.
Jafar: Remember, take nothing but the lamp.

Genie: Where’s your boss?
Aladdin: Um, my boss?
Genie: Look, kid, I’ve been doing this a long time, alright? There’s always a guy, you know. He’s cheated somebody, or buried somebody, or, I mean, you get my point. Where’s that guy?
Aladdin: I know that guy. He’s outside.

Genie: So here’s the basics. Step one, rub the lamp. Step two, say what you want. Step three, there is no step three. See? It’s that easy. You get three wishes. They must begin with rubbing the lamp and saying, “I wish.” Got it?

 I promise you, there’s not enough money and power on earth for you to be satisfied. Good?


Aladdin: Hey, can you make me a prince?
Genie: There’s a lot of gray area in “make me a prince”. I could just make you a prince.
[he uses his magic to create a prince]
Aladdin: Oh, no.
Genie: But you’d be snuggled up with that dude for the rest of your life.
[from the distance]
Prince: Yoh! Y’all seen my palace?


Genie: Stop, stop, stop. I made you look like a prince on the outside, but I didn’t change anything on the inside. Prince Ali got you to the door, but Aladdin has to open it.

Genie: It’s showtime.
Aladdin: No, I’m waiting for the right moment.
Genie: No, no. No waiting. We’re done waiting.
Aladdin: No. I’m in charge, okay? I say when it’s the right moment.

Genie: Wow. I mean, genie magic is really just a facade. At some point, the real character’s always going to shine through. But that’s a good thing, right? Now, she knows.

Jafar: As the old man said, “You should have left Agrabah when you had the chance.” I told you before to think bigger. You could have been the most powerful man in the room. But now, I hold the lamp. I hold the power.
Aladdin: You can’t find what you’re looking for in that lamp, Jafar. I tried and failed, and so will you.
Jafar: You think so? But I am Sultan! I am the greatest sorcerer the world has ever seen. I will create an empire that history cannot ignore. I can destroy cities. I can destroy kingdoms. And I can destroy you.
Aladdin: True. But who made you a sultan? Who made you a sorcerer? There will always be some thing, some man, some being more powerful than you.
Genie: What are you doing?
Aladdin: Genie gave you your power, and he can take it away.
Jafar: He serves me!
Aladdin: For now. But you’ll never have more power than the genie. You said it yourself, you’re either the most powerful in the room, or you’re nothing. You will always be second.
Iago: Second. Second.
Jafar: Second?! Only second? He serves me! I will make sure no one will ever say these words again!


Jafar: I will not forget you, boy! Mark my words. I will not forget what you have done to me.
Iago: Goodbye, Jafar.


Genie: Alright, last wish. Let’s get it.
Aladdin: Okay. Last wish. Genie…
Genie: I’m ready. Hold on. Here we go.
Aladdin: I wish…
Genie: The third and final wish.
Aladdin: I wish to set you free.
Genie: What? Woh. Oh! Wait.
[his genie shackels starts to disappear and becomes human and free]
Genie: Am I… Wait, wait, wait. Um, tell me to do something.
Aladdin: Um, give me some jams.
Genie: Get it yourself? Get your own jams!
[Aladdin and Genie embrace each other]
Genie: Thank you. Thank you.
Aladdin: No. Thank you, Genie. I owe you everything.

Sultan: Sit with me, my child. I’m sorry.
Jasmine: Baba, why are you…?
Sultan: Please, let me finish. I feared losing you, like I lost your mother. All I saw was my little girl, not the woman you have become. You have shown me courage and strength. You are the future of Agrabah. You shall be the next sultan.
[he kisses her hand]
Jasmine: Thank you, Baba.
Sultan: As sultan, you may change the law.
[referring to Aladdin]
Sultan: He is a good man.
[Jamsmine kisses his forehead]

Transcript: Greta Thunberg's Speech At The U.N. Climate Action Summit

Climate activist Greta Thunberg, 16, addressed the U.N.'s Climate Action Summit in New York City on Monday. Here's the full transcript of Thunberg's speech, beginning with her response to a question about the message she has for world leaders.

"My message is that we'll be watching you.

"This is all wrong. I shouldn't be up here. I should be back in school on the other side of the ocean. Yet you all come to us young people for hope. How dare you!

"You have stolen my dreams and my childhood with your empty words. And yet I'm one of the lucky ones. People are suffering. People are dying. Entire ecosystems are collapsing. We are in the beginning of a mass extinction, and all you can talk about is money and fairy tales of eternal economic growth. How dare you!

"For more than 30 years, the science has been crystal clear. How dare you continue to look away and come here saying that you're doing enough, when the politics and solutions needed are still nowhere in sight.

"You say you hear us and that you understand the urgency. But no matter how sad and angry I am, I do not want to believe that. Because if you really understood the situation and still kept on failing to act, then you would be evil. And that I refuse to believe.

"The popular idea of cutting our emissions in half in 10 years only gives us a 50% chance of staying below 1.5 degrees [Celsius], and the risk of setting off irreversible chain reactions beyond human control.

"Fifty percent may be acceptable to you. But those numbers do not include tipping points, most feedback loops, additional warming hidden by toxic air pollution or the aspects of equity and climate justice. They also rely on my generation sucking hundreds of billions of tons of your CO2 out of the air with technologies that barely exist.

"So a 50% risk is simply not acceptable to us — we who have to live with the consequences.

"To have a 67% chance of staying below a 1.5 degrees global temperature rise – the best odds given by the [Intergovernmental Panel on Climate Change] – the world had 420 gigatons of CO2 left to emit back on Jan. 1st, 2018. Today that figure is already down to less than 350 gigatons.

"How dare you pretend that this can be solved with just 'business as usual' and some technical solutions? With today's emissions levels, that remaining CO2 budget will be entirely gone within less than 8 1/2 years.

"There will not be any solutions or plans presented in line with these figures here today, because these numbers are too uncomfortable. And you are still not mature enough to tell it like it is.

"You are failing us. But the young people are starting to understand your betrayal. The eyes of all future generations are upon you. And if you choose to fail us, I say: We will never forgive you.

"We will not let you get away with this. Right here, right now is where we draw the line. The world is waking up. And change is coming, whether you like it or not.

"Thank you."

https://www.npr.org/2019/09/23/763452863/transcript-greta-thunbergs-speech-at-the-u-n-climate-action-summit

How to count the number of non-English character

#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; binmode STDOUT, 'utf8';
open(DATA, shift); binmode DATA, 'encoding(utf-8)'; while (<DATA>) { chomp; s/[A-Za-z]//g; say $_, ' ', length; }

reference:

https://www.perlmonks.org/?node_id=1084035

LSTM notes

https://colah.github.io/posts/2015-08-Understanding-LSTMs/

Understanding LSTM Networks

Posted on August 27, 2015

Recurrent Neural Networks

Humans don’t start their thinking from scratch every second. As you read this essay, you understand each word based on your understanding of previous words. You don’t throw everything away and start thinking from scratch again. Your thoughts have persistence.

Traditional neural networks can’t do this, and it seems like a major shortcoming. For example, imagine you want to classify what kind of event is happening at every point in a movie. It’s unclear how a traditional neural network could use its reasoning about previous events in the film to inform later ones.

Recurrent neural networks address this issue. They are networks with loops in them, allowing information to persist.

Recurrent Neural Networks have loops.

In the above diagram, a chunk of neural network, A$A$, looks at some input xt$x_t$ and outputs a value ht$h_t$. A loop allows information to be passed from one step of the network to the next.

These loops make recurrent neural networks seem kind of mysterious. However, if you think a bit more, it turns out that they aren’t all that different than a normal neural network. A recurrent neural network can be thought of as multiple copies of the same network, each passing a message to a successor. Consider what happens if we unroll the loop:

An unrolled recurrent neural network.

This chain-like nature reveals that recurrent neural networks are intimately related to sequences and lists. They’re the natural architecture of neural network to use for such data.

And they certainly are used! In the last few years, there have been incredible success applying RNNs to a variety of problems: speech recognition, language modeling, translation, image captioning… The list goes on. I’ll leave discussion of the amazing feats one can achieve with RNNs to Andrej Karpathy’s excellent blog post, The Unreasonable Effectiveness of Recurrent Neural Networks. But they really are pretty amazing.

Essential to these successes is the use of “LSTMs,” a very special kind of recurrent neural network which works, for many tasks, much much better than the standard version. Almost all exciting results based on recurrent neural networks are achieved with them. It’s these LSTMs that this essay will explore.

The Problem of Long-Term Dependencies

One of the appeals of RNNs is the idea that they might be able to connect previous information to the present task, such as using previous video frames might inform the understanding of the present frame. If RNNs could do this, they’d be extremely useful. But can they? It depends.

Sometimes, we only need to look at recent information to perform the present task. For example, consider a language model trying to predict the next word based on the previous ones. If we are trying to predict the last word in “the clouds are in the sky,” we don’t need any further context – it’s pretty obvious the next word is going to be sky. In such cases, where the gap between the relevant information and the place that it’s needed is small, RNNs can learn to use the past information.

But there are also cases where we need more context. Consider trying to predict the last word in the text “I grew up in France… I speak fluent French.” Recent information suggests that the next word is probably the name of a language, but if we want to narrow down which language, we need the context of France, from further back. It’s entirely possible for the gap between the relevant information and the point where it is needed to become very large.

Unfortunately, as that gap grows, RNNs become unable to learn to connect the information.

In theory, RNNs are absolutely capable of handling such “long-term dependencies.” A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don’t seem to be able to learn them. The problem was explored in depth by Hochreiter (1991) [German] and Bengio, et al. (1994), who found some pretty fundamental reasons why it might be difficult.

Thankfully, LSTMs don’t have this problem!

LSTM Networks

Long Short Term Memory networks – usually just called “LSTMs” – are a special kind of RNN, capable of learning long-term dependencies. They were introduced by Hochreiter & Schmidhuber (1997), and were refined and popularized by many people in following work.1 They work tremendously well on a large variety of problems, and are now widely used.

LSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!

All recurrent neural networks have the form of a chain of repeating modules of neural network. In standard RNNs, this repeating module will have a very simple structure, such as a single tanh layer.

The repeating module in a standard RNN contains a single layer.

LSTMs also have this chain like structure, but the repeating module has a different structure. Instead of having a single neural network layer, there are four, interacting in a very special way.

The repeating module in an LSTM contains four interacting layers.

Don’t worry about the details of what’s going on. We’ll walk through the LSTM diagram step by step later. For now, let’s just try to get comfortable with the notation we’ll be using.

In the above diagram, each line carries an entire vector, from the output of one node to the inputs of others. The pink circles represent pointwise operations, like vector addition, while the yellow boxes are learned neural network layers. Lines merging denote concatenation, while a line forking denote its content being copied and the copies going to different locations.

The Core Idea Behind LSTMs

The key to LSTMs is the cell state, the horizontal line running through the top of the diagram.

The cell state is kind of like a conveyor belt. It runs straight down the entire chain, with only some minor linear interactions. It’s very easy for information to just flow along it unchanged.

The LSTM does have the ability to remove or add information to the cell state, carefully regulated by structures called gates.

Gates are a way to optionally let information through. They are composed out of a sigmoid neural net layer and a pointwise multiplication operation.

The sigmoid layer outputs numbers between zero and one, describing how much of each component should be let through. A value of zero means “let nothing through,” while a value of one means “let everything through!”

An LSTM has three of these gates, to protect and control the cell state.

Step-by-Step LSTM Walk Through

The first step in our LSTM is to decide what information we’re going to throw away from the cell state. This decision is made by a sigmoid layer called the “forget gate layer.” It looks at ht−1$h_{t-1}$and xt$x_t$, and outputs a number between 0$0$ and 1$1$ for each number in the cell state Ct−1$C_{t-1}$. A 1$1$represents “completely keep this” while a 0$0$ represents “completely get rid of this.”

Let’s go back to our example of a language model trying to predict the next word based on all the previous ones. In such a problem, the cell state might include the gender of the present subject, so that the correct pronouns can be used. When we see a new subject, we want to forget the gender of the old subject.

The next step is to decide what new information we’re going to store in the cell state. This has two parts. First, a sigmoid layer called the “input gate layer” decides which values we’ll update. Next, a tanh layer creates a vector of new candidate values, C~t${\tilde{C}}_t$, that could be added to the state. In the next step, we’ll combine these two to create an update to the state.

In the example of our language model, we’d want to add the gender of the new subject to the cell state, to replace the old one we’re forgetting.

Eq. 4.

It’s now time to update the old cell state, Ct−1$C_{t-1}$, into the new cell state Ct$C_t$. The previous steps already decided what to do, we just need to actually do it.

We multiply the old state by ft$f_t$, forgetting the things we decided to forget earlier. Then we add it∗C~t$i_t\ast {\tilde{C}}_t$. This is the new candidate values, scaled by how much we decided to update each state value.

In the case of the language model, this is where we’d actually drop the information about the old subject’s gender and add the new information, as we decided in the previous steps.

Eq. 3. (Cell updating equation), f_t, i_t are the forgetting gates and inputting gates at time t

Finally, we need to decide what we’re going to output. This output will be based on our cell state, but will be a filtered version. First, we run a sigmoid layer which decides what parts of the cell state we’re going to output. Then, we put the cell state through tanh$\tanh$ (to push the values to be between −1$-1$ and 1$1$) and multiply it by the output of the sigmoid gate, so that we only output the parts we decided to.

For the language model example, since it just saw a subject, it might want to output information relevant to a verb, in case that’s what is coming next. For example, it might output whether the subject is singular or plural, so that we know what form a verb should be conjugated into if that’s what follows next.

Eq. 5.

Variants on Long Short Term Memory

What I’ve described so far is a pretty normal LSTM. But not all LSTMs are the same as the above. In fact, it seems like almost every paper involving LSTMs uses a slightly different version. The differences are minor, but it’s worth mentioning some of them.

One popular LSTM variant, introduced by Gers & Schmidhuber (2000), is adding “peephole connections.” This means that we let the gate layers look at the cell state.

The above diagram adds peepholes to all the gates, but many papers will give some peepholes and not others.

Another variation is to use coupled forget and input gates. Instead of separately deciding what to forget and what we should add new information to, we make those decisions together. We only forget when we’re going to input something in its place. We only input new values to the state when we forget something older.

A slightly more dramatic variation on the LSTM is the Gated Recurrent Unit, or GRU, introduced by Cho, et al. (2014). It combines the forget and input gates into a single “update gate.” It also merges the cell state and hidden state, and makes some other changes. The resulting model is simpler than standard LSTM models, and has been growing increasingly popular.

These are only a few of the most notable LSTM variants. There are lots of others, like Depth Gated RNNs by Yao, et al. (2015). There’s also some completely different approach to tackling long-term dependencies, like Clockwork RNNs by Koutnik, et al. (2014).

Which of these variants is best? Do the differences matter? Greff, et al. (2015) do a nice comparison of popular variants, finding that they’re all about the same. Jozefowicz, et al. (2015)tested more than ten thousand RNN architectures, finding some that worked better than LSTMs on certain tasks.

Conclusion

Earlier, I mentioned the remarkable results people are achieving with RNNs. Essentially all of these are achieved using LSTMs. They really work a lot better for most tasks!

Written down as a set of equations, LSTMs look pretty intimidating. Hopefully, walking through them step by step in this essay has made them a bit more approachable.

LSTMs were a big step in what we can accomplish with RNNs. It’s natural to wonder: is there another big step? A common opinion among researchers is: “Yes! There is a next step and it’s attention!” The idea is to let every step of an RNN pick information to look at from some larger collection of information. For example, if you are using an RNN to create a caption describing an image, it might pick a part of the image to look at for every word it outputs. In fact, Xu, et al.(2015) do exactly this – it might be a fun starting point if you want to explore attention! There’s been a number of really exciting results using attention, and it seems like a lot more are around the corner…

Attention isn’t the only exciting thread in RNN research. For example, Grid LSTMs by Kalchbrenner, et al. (2015) seem extremely promising. Work using RNNs in generative models – such as Gregor, et al. (2015), Chung, et al. (2015), or Bayer & Osendorfer (2015) – also seems very interesting. The last few years have been an exciting time for recurrent neural networks, and the coming ones promise to only be more so!

Acknowledgments

I’m grateful to a number of people for helping me better understand LSTMs, commenting on the visualizations, and providing feedback on this post.

I’m very grateful to my colleagues at Google for their helpful feedback, especially Oriol Vinyals, Greg Corrado, Jon Shlens, Luke Vilnis, and Ilya Sutskever. I’m also thankful to many other friends and colleagues for taking the time to help me, including Dario Amodei, and Jacob Steinhardt. I’m especially thankful to Kyunghyun Cho for extremely thoughtful correspondence about my diagrams.

Before this post, I practiced explaining LSTMs during two seminar series I taught on neural networks. Thanks to everyone who participated in those for their patience with me, and for their feedback.

---

1. In addition to the original authors, a lot of people contributed to the modern LSTM. A non-comprehensive list is: Felix Gers, Fred Cummins, Santiago Fernandez, Justin Bayer, Daan Wierstra, Julian Togelius, Faustino Gomez, Matteo Gagliolo, and Alex Graves.↩