<commit_before># manifest-search

A Common Lisp library to full text index common lisp documentation.  
The goal is to be able to search all of the online documentation of 
a running lisp system effectively.

Currently searches are run interactively with results being printed
to the REPL (stdout).  Eventually I would like to integrate this 
more tightly to manifest.

## Rationale

I (like I am guessing many) have at least one giant-ball-of-mud
utility library with lots of interesting but not well organized and
perhaps poorly named functionality.  I hope to make this code easier 
to organize, use and find even in the face of poor naming (assuming
documentation was better than the name).

This library is a direct answer to a co-workers complaints of being
unable to find (and thus use) functionality from libraries he is
unfamiliar with.  Hopefully it will evolve into a tool that (with
manifest) allows one to search for topics and receive documentation
about relavant functions.


## Dependancies

 * Montezuma provides full text indexing
 * Manifest provides the documentation gathering facilities
 * Other utility libs

## Example

The following will index all loaded packages and then search for 
things relevant to plist

```
;; adds all documentation for the running lisp to the search index
MANIFEST-SEARCH> (index-packages) 

;; searches all docs for ones that MUST contain plist
MANIFEST-SEARCH> (search-manifest "+plist")

NET.ACCELERATION.UTILS:OBJECT->PLIST <FUNCTION : 1.785362> 
    Converts an objects direct slots to a plist of (reader-name value ...) pairs 

ALEXANDRIA.0.DEV:PLIST-ALIST <FUNCTION : 1.7305975> 
    Returns an association list containing the same keys and values as the
property list PLIST in the same order.

ALEXANDRIA.0.DEV:DELETE-FROM-PLIST <FUNCTION : 1.6973867> 
    Just like REMOVE-FROM-PLIST, but this version may destructively modify the
provided plist.

NET.ACCELERATION.UTILS:INTERN-PLIST-KEYS <FUNCTION : 1.6199379> 
    interns plist keys to the specified package and returns a
   new plist of those keys and the original values

ALEXANDRIA.0.DEV:REMOVE-FROM-PLIST <FUNCTION : 1.4572165> 
    Returns a propery-list with same keys and values as PLIST, except that keys
in the list designated by KEYS and values corresponding to them are removed.
The returned property-list may share structure with the PLIST, but PLIST is
not destructively modified. Keys are compared using EQ.

ALEXANDRIA.0.DEV:PLIST-HASH-TABLE <FUNCTION : 1.4063838> 
    Returns a hash table containing the keys and values of the property list
PLIST. Hash table is initialized using the HASH-TABLE-INITARGS.

CLOSER-COMMON-LISP:SYMBOL-PLIST <FUNCTION : 1.3472456> 
    Return SYMBOL's property list.

  ... <More Results Elided> ...

```

## TODO:
 * Saving/Loading the index
 * Refreshing data in the index when it changes or periodically
 * Produce manifest links
 * Add search screen to manifest that ties into this system

## License

```
;; Copyright (c) 2011 <NAME> , Acceleration.net http://www.acceleration.net
;; All rights reserved.
;;
;; Redistribution and use in source and binary forms, with or without
;; modification, are permitted provided that the following conditions are
;; met:
;;
;;  - Redistributions of source code must retain the above copyright
;;    notice, this list of conditions and the following disclaimer.
;;
;;  - Redistributions in binary form must reproduce the above copyright<commit_msg>README todo<commit_after># manifest-search

A Common Lisp library to full text index common lisp documentation.  
The goal is to be able to search all of the online documentation of 
a running lisp system effectively.

Currently searches are run interactively with results being printed
to the REPL (stdout).  Eventually I would like to integrate this 
more tightly to manifest.

## Rationale

I (like I am guessing many) have at least one giant-ball-of-mud
utility library with lots of interesting but not well organized and
perhaps poorly named functionality.  I hope to make this code easier 
to organize, use and find even in the face of poor naming (assuming
documentation was better than the name).

This library is a direct answer to a co-workers complaints of being
unable to find (and thus use) functionality from libraries he is
unfamiliar with.  Hopefully it will evolve into a tool that (with
manifest) allows one to search for topics and receive documentation
about relavant functions.


## Dependancies

 * Montezuma provides full text indexing
 * Manifest provides the documentation gathering facilities
 * Other utility libs

## Example

The following will index all loaded packages and then search for 
things relevant to plist

```
;; adds all documentation for the running lisp to the search index
MANIFEST-SEARCH> (index-packages) 

;; searches all docs for ones that MUST contain plist
MANIFEST-SEARCH> (search-manifest "+plist")

NET.ACCELERATION.UTILS:OBJECT->PLIST <FUNCTION : 1.785362> 
    Converts an objects direct slots to a plist of (reader-name value ...) pairs 

ALEXANDRIA.0.DEV:PLIST-ALIST <FUNCTION : 1.7305975> 
    Returns an association list containing the same keys and values as the
property list PLIST in the same order.

ALEXANDRIA.0.DEV:DELETE-FROM-PLIST <FUNCTION : 1.6973867> 
    Just like REMOVE-FROM-PLIST, but this version may destructively modify the
provided plist.

NET.ACCELERATION.UTILS:INTERN-PLIST-KEYS <FUNCTION : 1.6199379> 
    interns plist keys to the specified package and returns a
   new plist of those keys and the original values

ALEXANDRIA.0.DEV:REMOVE-FROM-PLIST <FUNCTION : 1.4572165> 
    Returns a propery-list with same keys and values as PLIST, except that keys
in the list designated by KEYS and values corresponding to them are removed.
The returned property-list may share structure with the PLIST, but PLIST is
not destructively modified. Keys are compared using EQ.

ALEXANDRIA.0.DEV:PLIST-HASH-TABLE <FUNCTION : 1.4063838> 
    Returns a hash table containing the keys and values of the property list
PLIST. Hash table is initialized using the HASH-TABLE-INITARGS.

CLOSER-COMMON-LISP:SYMBOL-PLIST <FUNCTION : 1.3472456> 
    Return SYMBOL's property list.

  ... <More Results Elided> ...

```

## TODO:
 * Saving/Loading the index
 * Refreshing data in the index when it changes or periodically
 * Produce manifest links
 * Add search screen to manifest that ties into this system
 * Add a documentation index into the quicklisp system so that 
   users can search through the documentation of systems they have
   not yet installed (and thus discover functionality they could use)

## License

```
;; Copyright (c) 2011 <NAME> , Acceleration.net http://www.acceleration.net
;; All rights reserved.
;;
;; Redistribution and use in source and binary forms, with or without
;; modification, are permitted provided that the following conditions are
;; met:
;;
;;  - Redistributions of source code must retain the above copyright
;;    notice, this list of conditions and the following disclaimer.
;;
;;  - Redistributions in binary form must reproduce the above copyright
{'fraction_non_alphanumeric': 0.05872412336290663, 'fraction_numerical': 0.017743979721166033, 'mean_word_length': 4.18772826880935, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1232507', 'n_tokens_mistral': 2030, 'n_tokens_neox': 1860, 'n_words': 938}