Tools

Clojure Journey III – Emacs for Beginners Course

Posted on by Otavio Valadares

On my last post I talked about why I chose Spacemacs to be my text editor while coding Clojure, installed it and made a nice first setup. In this post I’ll talk about some workflows/commands that I think is important to have a great and smooth coding.

This post probably will be one of these posts that keep opened in your browser and you read it everytime that you need to remember something.

Neotree

First let’s talk about our tree explorer, on the last post I installed the neotree that is similar to NERDTree(from vim), and started to using it.

The first thing that I did was to bind the open/close of it to F8 key with the fallowing code:

(global-set-key [f8] 'neotree-toggle)

Now just press F8 and your neotree will open.

Opening neotree

To navigate inside Neotree just use your moving keys (arrow keys or HJKL), if you have any question about it, you can just type ? and a nice window will be displayed with everything that you may need:

Neotree help page

And this image saved me from writing at least 500 words, this image has everything that you will need related to Neotree.

If you don’t want to open neotree with F8 you can open by pressing SPC + f + t.

Window

Split windows is a common move to every programmer, to split is easy, just press ESC to go back to visual mode (if not already), and press: SPC + w + / to split vertically or SPC + w + - to split horizontally, where SPC is your space key. If you don’t want to use SPC, M-m + number will work too.

M-<letter> stand for alt + m

Spliting Emacs Window

To navigate between window is simple, just press SPC and the number of the window that you want to go, like SPC + 2 to go to window 2. But how to know the window number?

Spacemacs window numberr

If you’re using spacemacs, your window number will be displayed by default on the bottom left corner of your window, as the pink arrows of the image is pointing. The current window will always be in different color (as the example, window 2).

Opening/Searching Files

Open using neotree

If you want to open a file inside a window, the easiest way is to open using Neotree, just go to the window that you want to have a file opened, open your neotree, navigate to the file and open it.

Opening file by Neotree
Project search aka Ctrl + P

But if you don’t want to search for the file in your neotree, you want something more like the famous ctrl + p command from other text editors? In spacemacs with helm (the setup that I’m using), it’s very simple, just type: SPC + p + f and a window will open, just write the name that you’re searching for, and press ENTER to open in the current window.

Searching for a file using spacemacs and helm projectile

Buffers

Now we know how to manage our windows, is time to understand and manage buffers! Technically… Windows displays emacs buffers!

Everything that you open inside emacs resides on what is called “buffer”, every code, really everything! And the plot is, if you close your file/window with the traditional :q! (if you use vim mode), this code isn’t close at all, the buffer still alive!

Too see your buffers just type C-c + b and a window will open with a list of all buffers.

C-c stands for ctrl + c.

List buffers

But this window is not very useful…If we want to open a buffer the best way is to use the buffer search (similar to the file search that we see previously), just press C-x + b and you’ll have a full interactive search for buffers:

Helm buffers find

Just search for the wanted buffer and press enter to open it. If you use C-c + o it will open in a splitted window. (If you just want to open in the current window, just type enter)

Search on spacemacs bufferr

A nice shortcut is to press SPC + tab to change to the last buffer!

All the buffers keep alive, ok! But how do we kill them? Just press C-x + k select the buffer and press enter!

Search on project

To search for a pattern across all the project is very easy, just type: SPC + / and start writing your pattern:

Global search spacemacs

If you press C-c + o like in other commands, it will open in a splitted window.

Replace

It has a nice built-in way to do search/replace inside project too, it ‘s simple, SPC + p + R (pay attention on capitalized R) . After this it will ask for a pattern to replace, and what to replace, after this it will open a window with the first match, press y if you want to replace and n if no. Do this for all matches.

Search and replace

Search for commands

Emacs comes with a global commands search, just type M-x` and write what you’re searching for.

OFF: Themes

If you want to install more themes is very easy! You can install a layer names themes-megapack (If you don’t know how to install layers read my last post) and you’ll have more than 100 themes to choose.

OFF: Games inside editor

Emacs comes with some games installed, just open the global search with M-x and write the name of one of the games, like, tetris!

Tetris on emacs

Remember: With great power comes great responsibility.

Conclusions

This is a short and beginner-friendly emacs tutorial with all flows that I think is important to get started in a new text editor.

I hope I haven’t forgotten anything important to get started with it.

On the next posts related to Clojure, I’ll be using my spacemacs as my text editor, I’m loving to use it and I want that everyone that trries to start using it, have a great experience too.

Final thought

If you have any questions that I can help you with, please ask! Send an email (otaviopvaladares at gmail.com), pm me on my Twitter or comment on this post!

Follow my blog to get notified every new post:

Check my last posts:

Clojure Journey II – Chose a editor

Posted on by Otavio Valadares

My second step through my journey to learn Clojure is to setup my text editor or IDE. I’m not a evangelist of any text editor or IDE, so I started to search for what people usually use or is best to use while coding Clojure. After few minutes and several posts around the internet, I noticed the four most indicated tools:

I started to try each one.. First I removed Cursive + IntelliJ from my options because cursive is a paid plugin, and I didn’t tried it at all. Then I tried to install Nightcode, it looks amazing, but I don’t like to use IDEs, is my personal option and in a few cases I chose to use IDEs (Java and Go).

Update: @tanrax made a great comment in this post saying that Cursive plugin is free for open source contributors and students, if you’re one, check it!

Now I have two rivals text editors and I need to chose one. I started to take a look at fireplace plugin to vim, because I use my neovim on 80% of my hours of coding, but after installed it, I faced problems when trying to fix them.

Finally I decided to take a look at Emacs (Yes, it was my last option), I never tried it before because it looks scary at the first time. I’m learning a new language, why don’t learn a new editor? I give a chance to emacs and…. It looks amazing!

In this post I’ll describe my setup and how to use emacs to development with Clojure.

Emacs or Spacemacs?

If you already use emacs and have your custom setup you can skip this part aand use your setup.

But what about those who don’t use emacs?

Emacs or Spacemacs is a long discussion, and I don’t want to get on one side now. Simplifying, spacemacs is a “comunity-driven emacs” and it comes with a lot of configuration already made. If you chose to proceed with emacs you’ll spend a time searching for plugins and custom setups that are already done or easy done with space.

As I am a beginner user with emacs, I decided to proceed with spacemacs.

Installing Emacs

Before install spacemacs, first we need emacs installed.

To install emacs is very simple and they have a nice webpage with all the links needed. As a GNU/Linux user I just downloaded the tarball file of version 27.1, extracted it and compiled/installed it from the source using the traditional ./configure, make and make install. After this emacs is installed! 

$ emacs --version
GNU Emacs 27.1
Copyright (C) 2020 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file 
named COPYING.

If you have any trouble, you can see the INSTALL file that come with.

Installing spacemacs

Spacemacs have a beautiful website that need to be checked, but it has a nice README on github too.

As the date of this post, install spacemacs is very easy, just clone it in the correct directory:

git clone https://github.com/syl20bnr/spacemacs ~/.emacs.d

After this if you start your emacs, it should open spacemacs, and it will start to self setup. It will ask you some questions, but I chose the default one for every question, and selected the vim mode, to use vim bindings inside emacs. It helped me a lot, because I’m used to use vim.

Custom setup

Now you have a file named .spacemacs on your user directory, and this file is your configuration file, similar to other text editors.

The first thing that I noticed when I started it for the first time was the size of the font, it looks very small on my monitor. To change this is easy, open your .spacemacs file (Using your spacemacs, right?), and change your size:

dotspacemacs-default-font '("Source Code Pro"
                               :size 19
                               :weight normal
                               :width normal
                               :powerline-scale 1.1)

After this next time that you open spacemacs, your text size will be better.

Other thing that I missed from vim/nvim was the famous NERDTree plugin that is a file system explorer. Fortunately I found a layer named Neotree that is basically the same thing.

To install it is simple, as the documentation says, just add the layer name neotree to the dotspacemacs-configuration-layers list.

dotspacemacs-configuration-layers
   '(
     helm
     emacs-lisp
     neotree
     )

And if you want to open the tree pressing one ke, you can paste the fallowing code inside your config file:

(global-set-key [f8] 'neotree-toggle)

And if you like to have a system explorer looking like the vim’s one, just put this in your config file:

(setq neo-theme 'nerd)

And not, just reload your emacs:

The the only thing missing in our editor now is the Clojure support, and to install it is simple, just go to the same place where you installed the neotree layer, and write “clojure”, re-open your editor, and it’s done! The Clojure support for emacs is amazing and we’lll see in the next post!

What is spacemacs layers?

The first thing to clarify is that layers is a concept from spacemacs and not from emacs.

A layer can be defined as a collection of packages and it’s configuration required to make them work well with spacemacs and each other. You can easily find a open-source layer for almost everything that you’ll need (almost), but of course, you can make private layers too.

To install a layer is easy, just follow the same steps of this post: Add its name to dotspacemacs-configuration-layers list. You can check it’s documentation to learn how to use them.

Conclusions

In this post I chose my text editor that I’ll use when coding Clojure. On the next post we’ll talk about CIDER, The Clojure Interactive Development Environment for Emacs! And spoiler: It’s amazing.

Final thought

If you have any questions that I can help you with, please ask! Send an email (otaviopvaladares@gmail.com), pm me on my Twitter or comment on this post!

Types of storage on AWS S3

Posted on by Otavio Valadares

Amazon web services aka AWS provides a few ways of storing your files using the popular service S3(Simple Storage Service), the most common is the called S3 Standard that I think most people use on a normal day to day workflow, another common storage class is the famous S3 Glacier, but I don’t think most people know what is Standard-IA storage class, or One Zone-IA, in this post I want to write about each one and its uses cases.

It’s important to emphasize that this post is dated December 2018 and AWS is constantly changing and adding new features and probably, storage classes, if this post is outdated, please comment or check at my blog if you find a newer one.

All prices will be based on us-east-1 region.

Amazon S3 Standard

This is the default storage class, it’s designed for frequently accessed data, you can use this class for many things that will be accessed frequently, like, images that your user uploads on your website to use as profile image, or high-usage spreadsheets, other use cases can be: cloud applications, dynamic websites, content distribution, mobile and gaming applications, and big data analytics.

Key Features

As the aws official website say, this is the key features of standard class:

  • Low latency and high throughput performance (This is why it’s very recommended for high-usage data)
  • Designed for durability of 99.999999999% of objects across multiple Availability Zones (If you have 100 billion objects, Amazon ensures that you lost only one a year)
  • Resilient against events that impact an entire Availability Zone
  • Designed for 99.99% availability over a given year (Almost 52m 26s downtime per year)
  • Backed with the Amazon S3 Service Level Agreement for availability
  • Supports SSL for data in transit and encryption of data at rest
  • S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes

Pricing

Like almost all services provided, S3 has a long table of pricing separated by store amount, requests, data transfer, management, and transfer acceleration. You can check the full table here.

The S3 Standard Storage is the cheaper storage class when talking about request pricing and this is the main reason why you should use default S3 storage when you’re working with heavy access, you pay a nice price like $0.0004 per 1.000 GET requests.

At the end of the text I’ll make a better comparison between each type and pricing.

Amazon S3 Standard-Infrequent Access (S3 Standard-IA)

Check my last posts:

S3 Standard-IA is for data that is accessed less frequently, but when required needs fast access. You have the minimum storage period of 30 days and a minimum object size of 128KB. It is recommended for backups or logs(depending on the size of your logs, of course!).

Summarizing: All data that will not be accessed once a month and when required needs to be recovered just in time, you use S3 Standard-IA. Obviously, the problem is that you need to know previously the access pattern of your object, but as we’ll see later, this problem is already solved by Amazon S3 Intelligent-Tiering.

Key Features

  • Same low latency and high throughput performance of S3 Standard
  • Designed for durability of 99.999999999% of objects across multiple Availability Zones
  • Resilient against events that impact an entire Availability Zone
  • Data is resilient in the event of one entire Availability Zone destruction
  • Designed for 99.9% availability over a given year
  • Backed with the Amazon S3 Service Level Agreement for availability
  • Supports SSL for data in transit and encryption of data at rest
  • S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes

Pricing

The key point about pricing of S3 Standard-IA is that: Storage is cheap if you compare S3 standard, but a request is expensive.

Storage Price
All storage / Month $0.0125 per GB
Requests Price
Data Retrievals $0.01 per GB
Data Returned by S3 Select $0.01 per GB
Data Scanned by S3 Select $0.01 per GB
PUT, COPY, POST or LIST Request $0.01 per 1,000 requests
GET, SELECT and all other requests $0.001 per 1,000 requests

Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)

This is the younger brother of S3 Standard-IA and is almost the same service, the difference is that it stores your data at only one AZ (Availability Zone), unlike other S3 storage classes which store data in a minimum of three. With this trade-off, it costs 20% less than its older brother. It’s ideal for customers who want a lower-cost option for infrequently accesses data.

Key Features

The key features are the same as S3 Standard-IA, with the difference in the availability of 99.5% (Compared with 99.9% of Standard-IA).

Princing

The request price is the same as Standard-IA, and the storage price is 20% cheaper.

Storage Price
All storage / Month $0.01 per GB

Amazon S3 Intelligent-Tiering

This is the most interesting and newer one, it was released at November 2018, we saw that for high-usage files is better to use S3 Standard and to infrequent access data use Standard-IA, this is nice but we have one problem that some users already know, what we do if we don’t know the access pattern of the object? or if we don’t feel confident to change one object of the standard to Standard-IA?

Now imagine if Amazon creates something to do that for you? And they did it, and the problem has gone, this creation is the new Amazon S3 Intelligent-Tiering, it is designed for who want to optimize storage costs automatically when data access patterns change or you don’t know it.

It works with two objects tiers, one tier is similar do S3 Standard and another tier that is similar to Standard-IA, all object, when uploaded, is moved to the first tier that is for frequent access data, if the object has not been accessed for 30 consecutive days, Amazon moves it to the second tier that is for infrequent access data. Amazing, isn’t it?

Objects smaller than 128KB will never be transitioned to the infrequent access tier.

It’s important to point that we don’t have retrieval fees on this storage class.

Key Features

Follow my blog to get notified every new post:

  • Same low latency and high throughput performance of S3 Standard
  • Small monthly monitoring and auto-tiering fee
  • Automatically moves objects between two access tiers based on changing access patterns
  • Designed for durability of 99.999999999% of objects across multiple Availability Zones
  • Resilient against events that impact an entire Availability Zone
  • Designed for 99.9% availability over a given year
  • Backed with the Amazon S3 Service Level Agreement for availability
  • Supports SSL for data in transit and encryption of data at rest
  • S3 Lifecycle management for automatic migration of objects to other S3 Storage Classes

As you can see the key features are almost the same, except for the 99.9% availability (comparing with 99.99% of standard S3) and the feature of automatically moves the object between access tiers.

Princing

The price is awesome too when talking about storage pricing we have the same price of standard storage on frequent access tier, and the same price of Standard-IA to infrequent access tier.

Storage Price
Frequent Access Tier
————- —————
First 50 TB $0.023 per GB
Next 450 TB $0.022 per GB
Over 500 TB $0.021 per GB
————- —————
Frequent Access Tier
————- —————
All storage $0.0025 per 1,000 objects

For requests, we have the same price as the standard tier.

Requests Price
Data Returned by S3 Select $0.0007 per GB
Data Scanned by S3 Select $0.002 per GB
PUT, COPY, POST or LIST Request $0.0005 per 1,000 requests
GET, SELECT and all other requests $0.0004 per 1,000 requests

S3 Glacier

Glacier is designed to data archive if you have some data that you think you’ll never need again and if needed you can wait for almost 5 hours of the recovery process, go ahead and use Glacier, the price of the storage is the cheaper of the S3 classes. Example of things to store on S3 Glacier: Old logs that you can’t delete, old database dumps or your wedding pictures(just kidding but is a great idea).

One common action when working with Glacier is to setup a lifecycle policy to archive on Glacier data from S3 Standard bucket after n time, for example, 360 days, this is an amazing feature and provides a nice power to you.

Key Features

  • Designed for durability of 99.999999999% of objects across multiple Availability Zones
  • Data is resilient in the event of one entire Availability Zone destruction
  • Supports SSL for data in transit and encryption of data at rest
  • The low-cost design is ideal for long-term archive
  • Configurable retrieval times, from minutes to hours
  • S3 PUT API for direct uploads to S3 Glacier and S3 Lifecycle management for automatic migration of objects

Pricing

Amazon S3 Glacier has an extremely low cost of storage, cheaper than all other services that we studied.

Storage Price
All storage / Month $0.004 per GB

AWS provides three retrieval options, Expedited, Standard, Bulk, that range from a few minutes to hours and obviously from the cheapest to the most expensive one.

Expedited Standard Bulk
Retrieval Time 1-5 Minutes 3-5 hours
Data Retrievals $0.03 / GB $0.01 / GB
Retrievals Requests $0.01 / Request $0.050 / 1k Request

Conclusion

All storage class has its specific case of use, choose one used to be a problem when you don’t know the access pattern of the object, but today with S3 Intelligent-Tiering isn’t any more you can enable it and Amazon take a care for you, if add a lifecycle police to archive data from this bucket to Glacier after some time, you are with an amazing solution at your hands.

If you know the access pattern, just choose the better to your case and use it, S3 is amazing and cheap!

Final Thought

If you have any question that I can help you, please ask! Send an email (otaviopvaladares@gmail.com), pm me on my Twitter or comment on this post!

Links

AWS blog post about Intelligent Tiering

Annoucing S3 Intelligent-Tiering

S3 Pricing list

S3 Storage Classes

Glacier Retrieval Pricing Explained

How to Cut your S3 Cost in Half by Using the S3 Infrequent Access Storage Class

Migrating from vim to neovim

Posted on by Otavio Valadares

This week I decided to migrate to neovim, and give it a chance after a recommendation and some discussions about the future of vim text editor.

If you have a question about if you need to start using neovim I pretty recommend that you search on the internet and make your own decision. You can start by reading the following things:

Downloading

The first thing that I loved is the facility on download, at the official repository they provide a lot of good options for each OS, for Linux, an appimage file and that is awesome, for who doesn’t know it is like MAC OS applications file, for MAC OS you can download using brew and for windows you can download using chocolatey.

You can download neovim in the following two sites:

https://github.com/neovim/neovim/wiki/Installing-Neovim#install-from-download https://github.com/neovim/neovim/releases/

As I described I choose the appimage, so I just downloaded and moved it to /usr/local/bin and all works fine, now just type nvim and it will open.

Using the same setup of vim

The next question of everybody is probably “Ok, but I’ll need to configure everything again? All my plugins?” and the answer is No, you won’t.

If you’re coming from vim like me, you can simply type :help nvim-from-vim and follow the instructions, simplifying is just create a file ~/.config/nvim/init.vim that will be like your .vimrc, you’ll paste all your custom config at this file.

touch ~/.config/nvim/init.vim

If you’re a simple guy and just want to use your nvim in the same way as you use your vim, just paste the following commands at your init.vim that you created.

set runtimepath^=~/.vim runtimepath+=~/.vim/after
let &packpath = &runtimepath
source ~/.vimrc

It will route your nvim config to your previous vim setup, but it’s important to know that some plugins may not work, nvim isn’t entirely compatible with vim, but almost all plugins work on nvim too, with me, all plugins worked fine.

Different setup for nvim

For a lot of people the previous solution will be satisfying, but for me no. I want to stop using vim, so the previous solution doesn’t solve my problem, I want to have my own nvim setup. Luckily it’s easy too, on vim I use vim-pathogen to manage my plugins, so all my plugins are located at ~/.vim/bundle/, I just need to make a copy of this folder but on my nvim directory.

cp -r ~/.vim/bundle ~/.config/nvim/bundle/

And do the same with the .vimrc file.

cp ~/.vim/.vimrc ~/.config/nvim/init.vim

Now, if I want to modify something, I just need to modify on init.vim as you want, or install plugins at ~/.config/nvim/bundle.

Repository with nvim files

I like to create a Github repo to store my configs, so I just started a git repo at ~/.config/nvim and pushed it, you can find my nvim setup here.

You could find my vim files here but now with neovim, it’s deprecated.

Nvim differences

Neovim has some differences from vim, you can check it by typing :help vim-differences, but the major features (copying from the vim-differences output) is:

  • API
  • Lua scripting
  • Job control
  • Remote plugins
  • Providers
    • Clipboard
    • Node.js plugins
    • Python plugins
    • Ruby plugins
  • Shared data
  • Embedded terminal
  • VimL parser
  • XDG base directories

But I pretty recommend that you read the full documentation at :help vim-differences.

Nvim has some plugins that take advantage of specific nvim features, you can check it here.

Final thoughts

It wasn’t a very long post, and it’s purposeful, I don’t have much to talk about neovim yet, the objective of this post It only talks about how I migrate from vim to neovim. On next weeks I’ll write about my plugins and describing some tricks of my vim/nvim configs.

If you have any questions that I can help you with, please ask! Send an email (otaviopvaladares@gmail.com), pm me on my Twitter or comment on this post!

Follow my blog to get notified every new post:

vim-console

So, I started to use Vim

Posted on by Otavio Valadares

Since I started studying programming, I have heard about use Vim to code, but I have always used IDEs such as Visual Studio, CLion, IntelliJ, Eclipse, NetBeans, or text editor like Sublime and Atom (what I’m using in the last two years). When I had a look at vim tutors, or people using vim, I always thought, “That’s is so hard”, “I will never learn this”, “Look at this shortcuts”, “How I start using this?”, but in the last two months a several reasons made me to start learning and using Vim.

Vim is a light and fast text editor

Today almost all text editors like Atom with a tons of Js, sublime text and VS Code just fuck off your computer memory, and I’m not even going to talk about IDEs because anybody knows how much they consume your computer memory (And I’m not a person that is against using IDEs, in some kind of development it is good, like Android development for example).

You can check more on this fantastic post and in this repository.

Vim is a extensible text editor.

Everybody know, you can easily adapt vim for whatever you want to do, you can install the plugins that you want for power up your productivy.

Use everywhere

You can use Vim everywhere, yes, vim is cross-plataform, almost all operating systems supports vim, and others editor like nano no. And plus, you can use vim mode in a lot of editors/IDE’s like, emacs, sublime, xcode, eclipse etc.

Mainly because of these reasons I decided to Start using Vim.

Where did I started?

Like most of the people when I started, I decided to start using an already configurated Vim, like, YADR and The ultimate Vim configuration, these are awesome vim, I tried for few weeks on my personal projects, but when I was using them I felt lost in plugins, navigation, shortcuts and I have spent a lot of time searching on google how to do simple things, searching for understanding what that plugin does.

So I decided to start from zero installing Gvim (you can find the difference between gvim, vim-gtk, etc here) and searching for plugins that I need, the color scheme that I like among others, what happened? I started to like Vim, I started with raw Vim, and as necessary I went on installing my plugins and learned new tricks and shortcuts.

Which plugins have I installed on the beginning? Heres a list of them:

  • ctrlp.vim (I can’t live without ctrl+p command of other text editors, and heres a plugin to do this on vim)
  • nerdtree (The legendary tree explorer)
  • vim-polyglot (Collection of language packs for almost all languages)
  • grep.vim (Searching tool for Vim)
  • vim-rails (I installed this because I’m studyng Rails but its not a “Essencial plugin”)
  • fzd.vim (To use global search (Using the_silver_searcher))

I installed all these plugins using pathogen.vim because I found it more simple.

Finally, I spent less than an hour and already had a Vim to start coding and learning. After a few days I installed some other plugins, but for the start what I have mentioned is enough and Ipretty reccomend this aprproach for everyone that want to start and don’t know where or how to start. With this method you will learn the plugins you’re installing and the shortcut while you are working on your personal projects, and when you feel safe developing using Vim, you can start using it on your job.

If you want, you can watch some vim casts to improve your ability at http://vimcasts.org/, it’s very good quality.

If you want, you can find my vimfiles here, and its all documented.

Thanks.

That’s all for today folks, don’t forget to follow my blog and my twitter.