The Emacs Package Developer's Handbook

Table of Contents

After developing some little Emacs packages for a year or so, I began to notice how I'd forget little things that I learned, and then I'd have to go hunting for that information again. I also noticed how there are some issues for which there doesn't seem to be a "best practice" or "Standard Operating Procedure" to refer to.

So this is intended to be a place to collect and organize information related to Emacs package development. Built with Emacs, by Emacs package developers, for Emacs package developers.

You can read this Org file directly on the repository, or you can read the HTML version.

Note: The primary sections are listed at the top of the page in the horizontal bar.

Emacs Lisp   elisp

Buffers   buffers

Checkers / linters   linters checkers

TODO Flycheck-package

Collections   collections

Libraries   libraries

Destructuring   destructuring

See Pattern matching.

Editing   editing

Tools   tools

General   general

Libraries   libraries

  • Common Lisp Extensions (cl-lib)   built_in

    This is the built-in cl-lib package which implements Common Lisp functions and control structures for Emacs Lisp.

  • dash.el   dash

    Dash is a powerful general-purpose library that provides many useful functions and macros.

  • subr-x   built_in strings flow_control

    Less commonly used functions that complement basic APIs, often implemented in C code (like hash-tables and strings), and are not eligible for inclusion in subr.el.

    This is a built-in package that provides several useful functions and macros, such as thread-first / last, if-let / when-let, hash-table functions, and string functions. It's easy to forget about this, since:

    Do not document these functions in the lispref. http://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01006.html

Packaging   packaging

Best practices   best_practices

  • Autoloads   autoloads
    • TODO Autoloading macro-generated functions

      This may actually be a bug, or at least an unanswered question.

      How to use autoload cookies for custom defun-like macros? : emacs:

      Say I have a macro deffoo that expands to some custom kind of defun, and I want to use an autoload cookie to autoload the result. According to the manual, 

      ;;;###autoload (deffoo bar   ...)

      copies the entire form to autoloads.el, and something like 

      ;;;###autoload (autoload 'bar "this-file") (deffoo bar   ...)

      should be used instead. What confuses me is this StackOverflow comment by who appears to be Stefan Monnier, saying that Emacs should expand the macro before generating the autoload, and that it's probably a bug when this does not happen.

      Can anyone clear up what the intended behaviour is?

    • Articles   articles
  • Integration with other packages
    • Optional support

      Sometimes you want your package to integrate with other packages, but you don't want to require users to install those other packages. For example, you might want your package to work with Helm, Ivy, or the built-in Emacs completing-read, but you don't want to declare a dependency on and require Helm or Ivy, which would force users to install them to use your package.

      The best way to handle this is with the with-eval-after-load macro. The Emacs manual has a page on it, and this StackOverflow question has some more info. You can also see an example, which also uses declare-function to prevent byte-compiler errors.

  • Lexical binding   lexical_binding

    You should always use lexical binding by setting the header in the first line of the file: 

    ;;; filename.el --- File description  -*- lexical-binding: t; -*-
  • Skeleton

    Here is a skeleton you can use when starting new packages: 

    ;;; package-name.el --- Package description (don't include the word "Emacs")  -*- lexical-binding: t; -*-

;; Copyright (C) 2017 First Last

;; Author: First Last <name@example.com>
;; URL: http://example.com/package-name.el
;; Version: 0.1-pre
;; Package-Requires: ((emacs "25.2"))
;; Keywords: something

;;; Commentary:

;; This is my package.  It is nice.  You should try it.

;;;; Installation

;;;;; MELPA

;; If you installed from MELPA, you're done.

;;;;; Manual

;; Install these required packages:

;; + foo
;; + bar

;; Then put this file in your load-path, and put this in your init
;; file:

;; (require 'package-name)

;;;; Usage

;; Run one of these commands:

;; `package-name-command': Frobnicate the flange.

;;;; Tips

;; + You can customize settings in the `package-name' group.

;;;; Credits

;; This package would not have been possible without the following
;; packages: foo[1], which showed me how to bifurcate, and bar[2],
;; which takes care of flanges.
;;
;;  [1] https://example.com/foo.el
;;  [2] https://example.com/bar.el

;;; License:

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Code:

;;;; Requirements

(require 'foo)
(require 'bar)

;;;; Variables

(defgroup package-name nil
  "Settings for `package-name'."
  :link '(url-link "http://example.com/package-name.el"))

(defcustom package-name-something nil
  "This setting does something."
  :type 'something)

(defvar package-name-map
  (let ((map (copy-keymap keymap)))
    (define-key map (kbd "key") #'package-name-command)
    map)
  "Keymap for `package-name'.")

(defvar package-name-var nil
  "A variable.")

;;;; Functions

;;;;; Commands

;;;###autoload
(defun package-name-command (args)
  "Frobnicate the flange."
  (interactive)
  (package-name--something)
  (bar))

;;;;; Support

(defun package-name--something (args)
  "This function helps frobnicate the flange."
  (foo))

;;;; Footer

(provide 'package-name)

;;; package-name.el ends here

Reference   reference

  • Package headers and structure

    The Emacs manual gives this example (I've added the lexical-binding part). Also see Skeleton. 

    ;;; superfrobnicator.el --- Frobnicate and bifurcate flanges  -*- lexical-binding: t; -*-

;; Copyright (C) 2011 Free Software Foundation, Inc.

;; Author: J. R. Hacker <jrh@example.com>
;; Version: 1.3
;; Package-Requires: ((flange "1.0"))
;; Keywords: multimedia, frobnicate
;; URL: http://example.com/jrhacker/superfrobnicate

...

;;; Commentary:

;; This package provides a minor mode to frobnicate and/or
;; bifurcate any flanges you desire. To activate it, just type
...

;;;###autoload
(define-minor-mode superfrobnicator-mode
...

Pattern matching   destructuring pattern_matching

Articles   articles

  • Pattern Matching in Emacs Lisp – Wilfred Hughes::Blog   pcase shadchen cl dash

    Pattern matching is invaluable in elisp. Lists are ubiquitous, and a small amount of pattern matching can often replace a ton of verbose list fiddling.

    Since this is Lisp, we have lots of choices! In this post, we'll compare cl.el, pcase.el, dash.el, and shadchen, so you can choose the best fit for your project. We'll look at the most common use cases, and end with some recommendations.

    For the sake of this post, we'll consider both pattern matching and destructuring, as they're closely related concepts.

  • A callable plist data structure for Emacs   plists macros

    John Kitchin demonstrates some macros that make it easy to access plist values.

Libraries   libraries

Tools   tools

  • let-alist   alists macros destructuring

    let-alist is the best thing to happen to associative lists since the invention of the cons cell. This little macro lets you easily access the contents of an alist, concisely and efficiently, without having to specify them preemptively. It comes built-in with 25.1, and is also available on GNU Elpa for older Emacsen.

    Example: 

    (defun sx-question-list--print-info (question-data)
  "DOC"
  (let-alist question-data
    (list
     question-data
     (vector
      (int-to-string .score)
      (int-to-string .answer_count)
      .title " "
      .owner.display_name
      .last_activity_date sx-question-list-ago-string
      " " .tags))))
  • with-dict, with-plist-vals   macros plists

    Courtesy of John Kitchin:1 

    (defmacro with-dict (key-vals &rest body)
  "A context-manager for a plist where each key is a callable
function that returns the value."
  (declare (indent 1))
  (let* ((g (if (symbolp key-vals)
                (symbol-value key-vals)
              key-vals))
         (keys (-slice g 0 nil 2)))
    `(labels ,(loop for key in keys
                    collect
                    (list key '() `(plist-get ',g ,key)))
       ,@body)))

;; Used as:

(with-dict (:a 1 :b 'some-symbol :c 3)
           (:b))

(let ((d '(:key1 1 :key2 some-other-symbol :key3 3)))
  (with-dict d
             (format "We got %s" (:key2))))

    And: 

    (defmacro with-plist-vals (plist &rest body)
  "Bind the values of a plist to variables with the name of the keys."
  (declare (indent 1))
  `(let ,(loop for key in (-slice plist 0 nil 2)
               for val in (-slice plist 1 nil 2)
               collect (list (intern
                              (substring (symbol-name key) 1))
                             val))
     ,@body))

;; Used like:

(with-plist-vals (:a 4 :b 6)
                 (* 2 a))

Refactoring   refactoring

String manipulation   strings

Testing   testing

Libraries   libraries

Tools   tools

  • bench macro   macros

    From Phil Lord's m-buffer-el: 

    (defmacro bench (&rest body)
  (declare (indent defun))
  `(format "%e"
           (car
            (benchmark-run-compiled
                1000000
              (progn
                ,@body)))))

;; Use like this:
(bench
  (current-buffer)
  (point))
  • elp-profile macro   macros

    Call this macro from an Org source block and you'll get a results block showing which 20 functions were called the most times, how long they took to run, etc. Set prefixes to a list of strings matching the prefixes of the functions you want to instrument. 

    (defmacro elp-profile (times &rest body)
  (declare (indent defun))
  `(let ((prefixes '("string-" "s-" "buffer-" "append" "delq" "map"
                     "list" "car" "save-" "outline-" "delete-dups"
                     "sort" "line-" "nth" "concat" "char-to-string"
                     "rx-" "goto-" "when" "search-" "re-"))
         output)
     (dolist (prefix prefixes)
       (elp-instrument-package prefix))
     (dotimes (x ,times)
       ,@body)
     (elp-results)
     (elp-restore-all)
     (point-min)
     (forward-line 20)
     (delete-region (point) (point-max))
     (setq output (buffer-substring-no-properties (point-min) (point-max)))
     (kill-buffer)
     (delete-window)
     (let ((rows (s-lines output)))
       (append (list (list "Function" "Times called" "Total time" "Average time")
                     'hline)
               (cl-loop for row in rows
                        collect (s-split (rx (1+ space)) row 'omit-nulls))))))

;; Use like this:
(elp-profile 10
  (goto-char (point-min))
  (search-forward "something"))

Version control   version_control

Packages   packages

  • Magit   git

    One of the "killer apps" for Emacs–and for git!

XML / HTML   xml html

Libraries   libraries

  • esxml: An elisp library for working with xml, esxml and sxml

    Probably the most featureful, usable library at the moment.

    This library provides to formats for xml code generation. The primary form is esxml. esxml is the form that is returned by such functions as libxml-parse-xml-region and is used internally by emacs in many xml related libraries.

    It also provides esxml-query: 

    ;; Traditionally people pick one of the following options when faced
;; with the task of extracting data from XML in Emacs Lisp:
;;
;; - Using regular expressions on the unparsed document
;; - Manual tree traversal with `assoc', `car' and `cdr'
;;
;; Browsers faced a similar problem until jQuery happened, shortly
;; afterwards they started providing the `node.querySelector' and
;; `node.querySelectorAll' API for retrieving one or all nodes
;; matching a given CSS selector. This code implements the same API
;; with the `esxml-query' and `esxml-query-all' functions. The
;; following table summarizes the currently supported modifiers and
;; combinators:
;;
;; | Name                               | Supported? | Syntax      | 
;; |------------------------------------+------------+-------------|
;; | Namespaces                         | No         | foo|bar     | 
;; | Commas                             | Yes        | foo, bar    | 
;; | Descendant combinator              | Yes        | foo bar     | 
;; | Child combinator                   | Yes        | foo>bar     | 
;; | Adjacent sibling combinator        | No         | foo+bar     | 
;; | General sibling combinator         | No         | foo~bar     | 
;; | Universal selector                 | Yes        | *           | 
;; | Type selector                      | Yes        | tag         | 
;; | ID selector                        | Yes        | #foo        | 
;; | Class selector                     | Yes        | .foo        | 
;; | Attribute selector                 | Yes        | [foo]       | 
;; | Exact match attribute selector     | Yes        | [foo=bar]   | 
;; | Prefix match attribute selector    | Yes        | [foo^=bar]  | 
;; | Suffix match attribute selector    | Yes        | [foo$=bar]  | 
;; | Substring match attribute selector | Yes        | [foo*=bar]  | 
;; | Include match attribute selector   | Yes        | [foo~=bar]  | 
;; | Dash match attribute selector      | Yes        | [foo|=bar]  | 
;; | Attribute selector modifiers       | No         | [foo=bar i] | 
;; | Pseudo elements                    | No         | ::foo       | 
;; | Pseudo classes                     | No         | :foo        |

    Example: 

    (defun org-books--amazon (url)
  "Return plist of data for book at Amazon URL."
  (cl-flet ((field (target-field list)
                   (cl-loop for li in list
                            for (field value) = (ignore-errors
                                                  (-let (((_ _ (_ _ field) value) li))
                                                    (list field value)))
                            when (equal field target-field)
                            return (s-trim value))))
    (let* ((html (org-web-tools--get-url url))
           (tree (with-temp-buffer
                   (insert html)
                   (libxml-parse-html-region (point-min) (point-max))))
           (author (esxml-query "span.author a.contributorNameID *" tree))
           (title (esxml-query "div#booksTitle h1#title > span *" tree))
           (details (esxml-query-all "table#productDetailsTable ul li" tree))
           (date (if-let ((printed (third (esxml-query-all "div#booksTitle h1#title span *" tree))))
                     ;; Printed book
                     (s-replace "– " "" printed)
                   ;; Kindle book
                   (field "Publication Date:" details)))
           (asin (field "ASIN:" details))
           (publisher (-some->> (field "Publisher:" details)
                                (replace-regexp-in-string (rx " (" (1+ anything) ")") "")))
           (isbn-10 (field "ISBN-10:" details))
           (isbn-13 (field "ISBN-13:" details)))
      (list :author author :title title :publisher publisher :date date
            :asin asin :isbn-10 isbn-10 :isbn-13 isbn-13))))
  • elquery: Read and manipulate HTML

    It’s like jQuery, but way less useful.

    Example: 

    <html style="height: 100vh">
  <head class="kek"><title class="kek" data-bar="foo">Complex HTML Page</title></head>
  <body class="kek bur" style="height: 100%">
    <h1 id="bar" class="kek wow">Wow this is an example</h1>
    <input id="quux" class="kek foo"/>
    <iframe id="baz" sandbox="allow-same-origin allow-scripts allow-popups allow-forms"
            width="100%" height="100%" src="example.org">
    </iframe>
  </body>
</html>

    (let ((html (elq-read-file "~/kek.html")))
  (elq-el (car (elq-$ ".kek#quux" html))) ; => "input"
  (mapcar 'elq-el (elq-$ ".kek" html)) ; => ("input" "h1" "body" "title" "head")
  (mapcar (lambda (el) (elq-el (elq-parent el)))
          (elq-$ ".kek" html)) ; => ("body" "body" "html" "head" "html")
  (mapcar (lambda (el) (mapcar 'elq-el (elq-siblings el)))
          (elq-$ ".kek" html)) ; => (("h1" "input" "iframe") ("h1" "input" "iframe") ("head" "body") ("title") ("head" "body"))
  (elq-$ ".kek" html) ; => Hope you didn't like your messages buffer
  (elq-write html nil)) ; => "<html style=\"height: 100vh\"> ... </html>"
  • elfeed/xml-query.el

    Provides lisp-based (rather than string-based) selectors. This library is primarily aimed at internal elfeed use rather than general use, however it may be useful to others. The author is considering publishing it separately. 

    ;; This query grabs the top-level paragraph content from XHTML.

;; (xml-query-all '(html body p *) xhtml)

;; This query extracts all the links from an Atom feed.

;; (xml-query-all '(feed entry link [rel "alternate"] :href) xml)
  • enlive: query html document with css selectors

    This provides a limited set of lisp-based selectors (rather than string-based selectors).

    Example: 

    (require 'enlive)

(enlive-text
 (enlive-query (enlive-fetch "http://gnu.org/") [title])) ; => "The GNU Operating System and the Free Software Movement"
  • xml-plus: XML/HTML utilities

    Mostly undocumented, providing three main functions: 

    ;; Utility functions for xml parse trees.
;; - `xml+-query-all' and `xml+-query-first' are query functions that search
;; descendants in node lists. They don't work with namespace-aware parsing yet
;;
;; - `xml+-node-text' gets node text

Blogs   blogs

Planet Emacsen

This is the main community aggregator. You can find just about everyone's Emacs-related blog posts here.

Sacha Chua's Emacs News

This is Sacha's weekly Emacs news digest. Don't miss it!

Irreal

One of the top Emacs blogs, frequently updated, and often highlights other interesting blog entries in the community.

People   people

The Emacs community is so full of brilliant, generous people that I can't keep track of them all! I will surely overlook many, and I will add them in no particular order, but merely as I come across them again and again.

Artur Malabarba

Another prolific Emacs contributor, package developer, and blogger.

Damien Cassou

Packages

John Wiegley

John is the current Emacs maintainer.

Packages

Jonas Bernoulli

Jonas is a prolific Emacs package developer and maintainer. You could spend hours on his GitHub repo.

Packages

Matus Goljer

Packages

Oleh Krehel

Oleh is a prolific package author, having contributed many very high-quality packages. He also writes at his blog.

Sacha Chua

Sacha could easily be nominated the official Emacs ambassador, were there to be one. Her contributions to the Emacs and Org-mode communities are innumerable. One of her greatest recent contributions is her weekly Emacs news posts that serve as a digest of everything that happened in the Emacs world over the past week.

Wilfred Hughes

Wilfred has published several useful packages, and he's also leading the Rust Emacs port.

Contributions

Yes, please! Please send pull requests and file issues on the GitHub repo. This is intended to be a community project.

Tasks

TODO Articles to add [0/13]

TODO I wished GNU Emacs had… (2 min read)

TODO Reproduce bugs in emacs -Q (4 min read)

TODO Why package.el? (1 min read)

TODO Emacs script pitfalls (13 min read)

TODO Autoloads in Emacs Lisp (5 min read)

TODO Calling Python from Haskell (12 min read)

TODO Vincent Toups' projects

He has a lot of interesting libraries on his repo, and some of them are extensively documented. An aspiring Emacs Lisp developer could learn a lot from his code.

TODO Add MELPA

Mention @milkypostman, @purcell, @syohex, etc. Mention sandbox.

Code

This section contains code used to add to and update this document.

UNDERWAY Automate adding new links and summaries

TODO Get summary of page

DONE Get archive.is link for page

This bookmarklet should provide a way to get the URL: 

javascript:void(open('https://archive.today/?run=1&url='+encodeURIComponent(document.location)))

Seems to only work if run in a browser, with JavaScript. But there's a Python package that has a shell command. So: 

(require 's)

(defun emacs-package-dev-handbook--archiveis-capture (url)
  "Return archive.is archived URL for URL."
  (with-temp-buffer
    (when (zerop (call-process "archiveis" nil t nil url))
      (s-trim (buffer-string)))))

(cl-defun emacs-package-dev-handbook-insert-archiveis-property (&optional url)
  "Set the \"archive.is\" property for entry at point to the archived URL.
 Assumes heading on/before point is an Org link to a web page. If
 URL is given, archive that URL instead."
  (interactive)
  (when-let ((url (or url (save-excursion
                            (unless (org-at-heading-p)
                              (org-back-to-heading))
                            (beginning-of-line)
                            (when (re-search-forward org-bracket-link-regexp (line-end-position) 'noerror)
                              (org-link-unescape (match-string-no-properties 1))))))
             (archive-url (emacs-package-dev-handbook--archiveis-capture url)))
    (org-set-property "archive.is" archive-url)))

TODO Insert new entry at point

Maybe use capture templates and refile?

Table of Contents

Currently using org-make-toc, which is uploaded but unfinished and unpackaged.

Config

I love Emacs and Org mode. This makes it so easy to make the document…alive! And automated! Beautiful.

Footnotes:

1
Copyright by John Kitchin, licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Author: Adam Porter

Created: 2017-11-04 Sat 02:02

Validate