--name |
an optional name you can assign to your D-Tale instance (this will be displayed in the </code> & Instances popup)</td>
</tr>
<tr>
<td align="left"><code>--debug</code></td>
<td align="left">turn on Flask's "debug" mode for your D-Tale instance</td>
</tr>
<tr>
<td align="left"><code>--no-reaper</code></td>
<td align="left">flag to turn off auto-reaping subprocess (kill D-Tale instances after an hour of inactivity), good for long-running displays</td>
</tr>
<tr>
<td align="left"><code>--open-browser</code></td>
<td align="left">flag to automatically open up your server's default browser to your D-Tale instance</td>
</tr>
<tr>
<td align="left"><code>--force</code></td>
<td align="left">flag to force D-Tale to try an kill any pre-existing process at the port you've specified so it can use it</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">Loading data from <a href="/github.com/man-group/ArcticDB"><strong>ArcticDB</strong>(high performance, serverless DataFrame database)</a> (this requires either installing <strong>arcticdb</strong> or <strong>dtale[arcticdb]</strong>)</p>
<div dir="auto"><tt>dtale --arcticdb-uri lmdb:///<span><</span>path<span>></span> --arcticdb-library jdoe.my_lib --arcticdb-symbol my_symbol</tt></div>
<p dir="auto">If you would like to change your storage mechanism to ArcticDB then add the <code>--arcticdb-use_store</code> flag</p>
<div dir="auto"><tt>dtale --arcticdb-uri lmdb:///<span><</span>path<span>></span> --arcticdb-library my_lib --arcticdb-symbol my_symbol --arcticdb-use_store</tt></div>
<p dir="auto">Loading data from <a href="/github.com/man-group/arctic"><strong>arctic</strong>(high performance datastore for pandas dataframes)</a> (this requires either installing <strong>arctic</strong> or <strong>dtale[arctic]</strong>)</p>
<div dir="auto"><tt>dtale --arctic-uri mongodb://localhost:27027 --arctic-library my_lib --arctic-symbol my_symbol --arctic-start 20130101 --arctic-end 20161231</tt></div>
<p dir="auto">Loading data from <strong>CSV</strong></p>
<div dir="auto"><tt>dtale --csv-path /home/jdoe/my_csv.csv --csv-parse_dates date</tt></div>
<p dir="auto">Loading data from <strong>EXCEL</strong></p>
<div dir="auto"><tt>dtale --excel-path /home/jdoe/my_csv.xlsx --excel-parse_dates date<br>dtale --excel-path /home/jdoe/my_csv.xls --excel-parse_dates date</tt></div>
<p dir="auto">Loading data from <strong>JSON</strong></p>
<div dir="auto"><tt>dtale --json-path /home/jdoe/my_json.json --json-parse_dates date</tt></div>
<p dir="auto">or</p>
<div dir="auto"><tt>dtale --json-path http://json-endpoint --json-parse_dates date</tt></div>
<p dir="auto">Loading data from <strong>R Datasets</strong></p>
<div dir="auto"><tt>dtale --r-path /home/jdoe/my_dataset.rda</tt></div>
<p dir="auto">Loading data from <strong>SQLite DB Files</strong></p>
<div dir="auto"><tt>dtale --sqlite-path /home/jdoe/test.sqlite3 --sqlite-table test_table</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Custom Command-line Loaders</h3></div>
<p dir="auto">Loading data from a <strong>Custom</strong> loader</p>
<ul dir="auto">
<li>Using the DTALE_CLI_LOADERS environment variable, specify a path to a location containing some python modules</li>
<li>Any python module containing the global variables LOADER_KEY & LOADER_PROPS will be picked up as a custom loader
<ul dir="auto">
<li>LOADER_KEY: the key that will be associated with your loader. By default you are given <strong>arctic</strong> & <strong>csv</strong> (if you use one of these are your key it will override these)</li>
<li>LOADER_PROPS: the individual props available to be specified.
<ul dir="auto">
<li>For example, with arctic we have host, library, symbol, start & end.</li>
<li>If you leave this property as an empty list your loader will be treated as a flag. For example, instead of using all the arctic properties we would simply specify <code>--arctic</code> (this wouldn't work well in arctic's case since it depends on all those properties)</li>
</ul>
</li>
</ul>
</li>
<li>You will also need to specify a function with the following signature <code>def find_loader(kwargs)</code> which returns a function that returns a dataframe or <code>None</code></li>
<li>Here is an example of a custom loader:</li>
</ul>
<div dir="auto"''
IMPORTANT!!! This global variable is required for building any customized CLI loader.
When find loaders on startup it will search for any modules containing the global variable LOADER_KEY.
'''
LOADER_KEY = 'testdata'
LOADER_PROPS = ['rows', 'columns']
def test_data(rows, columns):
import pandas as pd
import numpy as np
import random
from past.utils import old_div
from pandas.tseries.offsets import Day
from dtale.utils import dict_merge
import string
now = pd.Timestamp(pd.Timestamp('now').date())
dates = pd.date_range(now - Day(364), now)
num_of_securities = max(old_div(rows, len(dates)), 1) # always have at least one security
securities = [
dict(security_id=100000 + sec_id, int_val=random.randint(1, 100000000000),
str_val=random.choice(string.ascii_letters) * 5)
for sec_id in range(num_of_securities)
]
data = pd.concat([
pd.DataFrame([dict_merge(dict(date=date), sd) for sd in securities])
for date in dates
], ignore_index=True)[['date', 'security_id', 'int_val', 'str_val']]
col_names = ['Col{}'.format(c) for c in range(columns)]
return pd.concat([data, pd.DataFrame(np.random.randn(len(data), columns), columns=col_names)], axis=1)
# IMPORTANT!!! This function is required for building any customized CLI loader.
def find_loader(kwargs):
test_data_opts = get_loader_options(LOADER_KEY, LOADER_PROPS, kwargs)
if len([f for f in test_data_opts.values() if f]):
def _testdata_loader():
return test_data(int(test_data_opts.get('rows', 1000500)), int(test_data_opts.get('columns', 96)))
return _testdata_loader
return None"><tt><span>from</span> <span>dtale</span>.<span>cli</span>.<span>clickutils</span> <span>import</span> <span>get_loader_options</span><br><br><span>'''</span><br><span> IMPORTANT!!! This global variable is required for building any customized CLI loader.</span><br><span> When find loaders on startup it will search for any modules containing the global variable LOADER_KEY.</span><br><span>'''</span><br><span>LOADER_KEY</span> <span>=</span> <span>'testdata'</span><br><span>LOADER_PROPS</span> <span>=</span> [<span>'rows'</span>, <span>'columns'</span>]<br><br><br><span>def</span> <span>test_data</span>(<span>rows</span>, <span>columns</span>):<br> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br> <span>import</span> <span>numpy</span> <span>as</span> <span>np</span><br> <span>import</span> <span>random</span><br> <span>from</span> <span>past</span>.<span>utils</span> <span>import</span> <span>old_div</span><br> <span>from</span> <span>pandas</span>.<span>tseries</span>.<span>offsets</span> <span>import</span> <span>Day</span><br> <span>from</span> <span>dtale</span>.<span>utils</span> <span>import</span> <span>dict_merge</span><br> <span>import</span> <span>string</span><br><br> <span>now</span> <span>=</span> <span>pd</span>.<span>Timestamp</span>(<span>pd</span>.<span>Timestamp</span>(<span>'now'</span>).<span>date</span>())<br> <span>dates</span> <span>=</span> <span>pd</span>.<span>date_range</span>(<span>now</span> <span>-</span> <span>Day</span>(<span>364</span>), <span>now</span>)<br> <span>num_of_securities</span> <span>=</span> <span>max</span>(<span>old_div</span>(<span>rows</span>, <span>len</span>(<span>dates</span>)), <span>1</span>) <span># always have at least one security</span><br> <span>securities</span> <span>=</span> [<br> <span>dict</span>(<span>security_id</span><span>=</span><span>100000</span> <span>+</span> <span>sec_id</span>, <span>int_val</span><span>=</span><span>random</span>.<span>randint</span>(<span>1</span>, <span>100000000000</span>),<br> <span>str_val</span><span>=</span><span>random</span>.<span>choice</span>(<span>string</span>.<span>ascii_letters</span>) <span>*</span> <span>5</span>)<br> <span>for</span> <span>sec_id</span> <span>in</span> <span>range</span>(<span>num_of_securities</span>)<br> ]<br> <span>data</span> <span>=</span> <span>pd</span>.<span>concat</span>([<br> <span>pd</span>.<span>DataFrame</span>([<span>dict_merge</span>(<span>dict</span>(<span>date</span><span>=</span><span>date</span>), <span>sd</span>) <span>for</span> <span>sd</span> <span>in</span> <span>securities</span>])<br> <span>for</span> <span>date</span> <span>in</span> <span>dates</span><br> ], <span>ignore_index</span><span>=</span><span>True</span>)[[<span>'date'</span>, <span>'security_id'</span>, <span>'int_val'</span>, <span>'str_val'</span>]]<br><br> <span>col_names</span> <span>=</span> [<span>'Col{}'</span>.<span>format</span>(<span>c</span>) <span>for</span> <span>c</span> <span>in</span> <span>range</span>(<span>columns</span>)]<br> <span>return</span> <span>pd</span>.<span>concat</span>([<span>data</span>, <span>pd</span>.<span>DataFrame</span>(<span>np</span>.<span>random</span>.<span>randn</span>(<span>len</span>(<span>data</span>), <span>columns</span>), <span>columns</span><span>=</span><span>col_names</span>)], <span>axis</span><span>=</span><span>1</span>)<br><br><br><span># IMPORTANT!!! This function is required for building any customized CLI loader.</span><br><span>def</span> <span>find_loader</span>(<span>kwargs</span>):<br> <span>test_data_opts</span> <span>=</span> <span>get_loader_options</span>(<span>LOADER_KEY</span>, <span>LOADER_PROPS</span>, <span>kwargs</span>)<br> <span>if</span> <span>len</span>([<span>f</span> <span>for</span> <span>f</span> <span>in</span> <span>test_data_opts</span>.<span>values</span>() <span>if</span> <span>f</span>]):<br> <span>def</span> <span>_testdata_loader</span>():<br> <span>return</span> <span>test_data</span>(<span>int</span>(<span>test_data_opts</span>.<span>get</span>(<span>'rows'</span>, <span>1000500</span>)), <span>int</span>(<span>test_data_opts</span>.<span>get</span>(<span>'columns'</span>, <span>96</span>)))<br><br> <span>return</span> <span>_testdata_loader</span><br> <span>return</span> <span>None</span></tt></div>
<p dir="auto">In this example we simplying building a dataframe with some dummy data based on dimensions specified on the command-line:</p>
<ul dir="auto">
<li><code>--testdata-rows</code></li>
<li><code>--testdata-columns</code></li>
</ul>
<p dir="auto">Here's how you would use this loader:</p>
<div dir="auto"dtale --testdata-rows 10 --testdata-columns 5'"><tt>DTALE_CLI_LOADERS=./path_to_loaders bash -c <span><span>'</span>dtale --testdata-rows 10 --testdata-columns 5<span>'</span></span></tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Authentication</h3></div>
<p dir="auto">You can choose to use optional authentication by adding the following to your D-Tale <code>.ini</code> file (<a href="/github.com/man-group/dtale/blob/master/docs/CONFIGURATION.md">directions here</a>):</p>
<div dir="auto"><tt><span>[auth]</span><br><span>active</span> = True<br><span>username</span> = johndoe<br><span>password</span> = 1337h4xOr</tt></div>
<p dir="auto">Or you can call the following:</p>
<div dir="auto"active': True, 'username': 'johndoe', 'password': '1337h4x0r'})"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><br><span>global_state</span>.<span>set_auth_settings</span>({<span>'active'</span>: <span>True</span>, <span>'username'</span>: <span>'johndoe'</span>, <span>'password'</span>: <span>'1337h4x0r'</span>})</tt></div>
<p dir="auto">If you have done this before initially starting D-Tale it will have authentication applied. If you are adding this after starting D-Tale you will have to kill your service and start it over.</p>
<p dir="auto">When opening your D-Tale session you will be presented with a screen like this:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/login.png"></a></p>
<p dir="auto">From there you can enter the credentials you either set in your <code>.ini</code> file or in your call to <code>dtale.global_state.set_auth_settings</code> and you will be brought to the main grid as normal. You will now have an additional option in your main menu to logout:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/logout.png"></a></p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Instance Settings</h3></div>
<p dir="auto">Users can set front-end properties on their instances programmatically in the <code>dtale.show</code> function or by calling the <code>update_settings</code> function on their instance. For example:</p>
<div dir="auto"a','b','c','d','e']
))
dtale.show(
df,
locked=['c'],
column_formats={'a': {'fmt': '0.0000'}},
nan_display='...',
background_mode='heatmap-col',
sort=[('a','DESC')],
vertical_headers=True,
)"><tt><span>import</span> <span>dtale</span><br><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><br><span>df</span> <span>=</span> <span>pd</span>.<span>DataFrame</span>(<span>dict</span>(<br> <span>a</span><span>=</span>[<span>1</span>,<span>2</span>,<span>3</span>,<span>4</span>,<span>5</span>],<br> <span>b</span><span>=</span>[<span>6</span>,<span>7</span>,<span>8</span>,<span>9</span>,<span>10</span>],<br> <span>c</span><span>=</span>[<span>'a'</span>,<span>'b'</span>,<span>'c'</span>,<span>'d'</span>,<span>'e'</span>]<br>))<br><span>dtale</span>.<span>show</span>(<br> <span>df</span>,<br> <span>locked</span><span>=</span>[<span>'c'</span>],<br> <span>column_formats</span><span>=</span>{<span>'a'</span>: {<span>'fmt'</span>: <span>'0.0000'</span>}},<br> <span>nan_display</span><span>=</span><span>'...'</span>,<br> <span>background_mode</span><span>=</span><span>'heatmap-col'</span>,<br> <span>sort</span><span>=</span>[(<span>'a'</span>,<span>'DESC'</span>)],<br> <span>vertical_headers</span><span>=</span><span>True</span>,<br>)</tt></div>
<p dir="auto">or</p>
<div dir="auto"a','b','c','d','e']
))
d = dtale.show(
df
)
d.update_settings(
locked=['c'],
column_formats={'a': {'fmt': '0.0000'}},
nan_display='...',
background_mode='heatmap-col',
sort=[('a','DESC')],
vertical_headers=True,
)
d"><tt><span>import</span> <span>dtale</span><br><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><br><span>df</span> <span>=</span> <span>pd</span>.<span>DataFrame</span>(<span>dict</span>(<br> <span>a</span><span>=</span>[<span>1</span>,<span>2</span>,<span>3</span>,<span>4</span>,<span>5</span>],<br> <span>b</span><span>=</span>[<span>6</span>,<span>7</span>,<span>8</span>,<span>9</span>,<span>10</span>],<br> <span>c</span><span>=</span>[<span>'a'</span>,<span>'b'</span>,<span>'c'</span>,<span>'d'</span>,<span>'e'</span>]<br>))<br><span>d</span> <span>=</span> <span>dtale</span>.<span>show</span>(<br> <span>df</span><br>)<br><span>d</span>.<span>update_settings</span>(<br> <span>locked</span><span>=</span>[<span>'c'</span>],<br> <span>column_formats</span><span>=</span>{<span>'a'</span>: {<span>'fmt'</span>: <span>'0.0000'</span>}},<br> <span>nan_display</span><span>=</span><span>'...'</span>,<br> <span>background_mode</span><span>=</span><span>'heatmap-col'</span>,<br> <span>sort</span><span>=</span>[(<span>'a'</span>,<span>'DESC'</span>)],<br> <span>vertical_headers</span><span>=</span><span>True</span>,<br>)<br><span>d</span></tt></div>
<p dir="auto">Here's a short description of each instance setting available:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="left">Option</th>
<th align="left">Type</th>
<th align="left">Default</th>
<th align="left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><code>show_columns</code></td>
<td align="left">list</td>
<td align="left"><code>None</code></td>
<td align="left">A list of column names you would like displayed in your grid. Anything else will be hidden.</td>
</tr>
<tr>
<td align="left"><code>hide_columns</code></td>
<td align="left">list</td>
<td align="left"><code>None</code></td>
<td align="left">A list of column names you would like initially hidden from the grid display.</td>
</tr>
<tr>
<td align="left"><code>column_formats</code></td>
<td align="left">dict</td>
<td align="left"><code>None</code></td>
<td align="left">A dictionary of column name keys and their front-end display configuration. See column_formats below.</td>
</tr>
<tr>
<td align="left"><code>nan_display</code></td>
<td align="left">str</td>
<td align="left"><code>None</code></td>
<td align="left">Converts any <code>nan</code> values in your dataframe to this when sent to the browser (doesn't change the dataframe).</td>
</tr>
<tr>
<td align="left"><code>sort</code></td>
<td align="left">list</td>
<td align="left"><code>None</code></td>
<td align="left">List of tuples which sort your dataframe. See sort below.</td>
</tr>
<tr>
<td align="left"><code>locked</code></td>
<td align="left">list</td>
<td align="left"><code>None</code></td>
<td align="left">List of column names which will be locked to the right side of your grid while you scroll to the left.</td>
</tr>
<tr>
<td align="left"><code>background_mode</code></td>
<td align="left">str</td>
<td align="left"><code>None</code></td>
<td align="left">A string denoting one of the background display modes. See background_mode below.</td>
</tr>
<tr>
<td align="left"><code>range_highlights</code></td>
<td align="left">dict</td>
<td align="left"><code>None</code></td>
<td align="left">Dictionary of column name keys and range configurations for conditional highlighting. See range_highlights below.</td>
</tr>
<tr>
<td align="left"><code>vertical_headers</code></td>
<td align="left">bool</td>
<td align="left"><code>False</code></td>
<td align="left">If <code>True</code>, headers in your grid will be rotated 90 degrees vertically to conserve width. See vertical_headers below.</td>
</tr>
<tr>
<td align="left"><code>precision</code></td>
<td align="left">int</td>
<td align="left"><code>2</code></td>
<td align="left">The default decimal precision for float values.</td>
</tr>
<tr>
<td align="left"><code>theme</code></td>
<td align="left">str</td>
<td align="left"><code>"light"</code></td>
<td align="left">Display theme for the app. Options are <code>"light"</code> and <code>"dark"</code>.</td>
</tr>
<tr>
<td align="left"><code>main_title</code></td>
<td align="left">str</td>
<td align="left"><code>None</code></td>
<td align="left">Custom title to display in the header and browser tab (replaces "D-Tale").</td>
</tr>
<tr>
<td align="left"><code>main_title_font</code></td>
<td align="left">str</td>
<td align="left"><code>None</code></td>
<td align="left">Custom font family to use for the main title (e.g., <code>"Arial"</code>, <code>"Helvetica"</code>).</td>
</tr>
<tr>
<td align="left"><code>auto_hide_empty_columns</code></td>
<td align="left">bool</td>
<td align="left"><code>False</code></td>
<td align="left">If <code>True</code>, auto-hide any columns comprised entirely of NaN values.</td>
</tr>
<tr>
<td align="left"><code>highlight_filter</code></td>
<td align="left">bool</td>
<td align="left"><code>False</code></td>
<td align="left">If <code>True</code>, highlight rows that match a filter rather than hiding them.</td>
</tr>
<tr>
<td align="left"><code>hide_shutdown</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the "Shutdown" button from users.</td>
</tr>
<tr>
<td align="left"><code>hide_header_editor</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the header editor when editing cells.</td>
</tr>
<tr>
<td align="left"><code>lock_header_menu</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, always display the header menu (normally only shows on hover).</td>
</tr>
<tr>
<td align="left"><code>hide_header_menu</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the header menu from the screen.</td>
</tr>
<tr>
<td align="left"><code>hide_main_menu</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the main menu from the screen.</td>
</tr>
<tr>
<td align="left"><code>hide_column_menus</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the column menus from the screen.</td>
</tr>
<tr>
<td align="left"><code>hide_row_expanders</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, hide the row expanders from the screen.</td>
</tr>
<tr>
<td align="left"><code>column_edit_options</code></td>
<td align="left">dict</td>
<td align="left"><code>None</code></td>
<td align="left">Options to allow on the front-end when editing a cell for specific columns. See column_edit_options below.</td>
</tr>
<tr>
<td align="left"><code>enable_custom_filters</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, enable users to create custom filters from the UI.</td>
</tr>
<tr>
<td align="left"><code>enable_web_uploads</code></td>
<td align="left">bool</td>
<td align="left"><code>None</code></td>
<td align="left">If <code>True</code>, enable users to upload files using URLs from the UI.</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">column_formats</h4></div>
<p dir="auto">A dictionary of column name keys and their front-end display configuration. Here are examples of the different format configurations:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="left">Data Type</th>
<th align="left">Format</th>
<th align="left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Numeric</td>
<td align="left"><code>{'fmt': '0.00000'}</code></td>
<td align="left">Format number with 5 decimal places</td>
</tr>
<tr>
<td align="left">String</td>
<td align="left"><code>{'fmt': {'truncate': 10}}</code></td>
<td align="left">Truncate string values to no more than 10 characters followed by an ellipsis</td>
</tr>
<tr>
<td align="left">String</td>
<td align="left"><code>{'fmt': {'link': True}}</code></td>
<td align="left">If your strings are URLs, convert them to clickable links</td>
</tr>
<tr>
<td align="left">String</td>
<td align="left"><code>{'fmt': {'html': True}}</code></td>
<td align="left">If your strings are HTML fragments, render them as HTML</td>
</tr>
<tr>
<td align="left">Date</td>
<td align="left"><code>{'fmt': 'MMMM Do YYYY, h:mm:ss a'}</code></td>
<td align="left">Uses <a href="/momentjs.com/docs/#/displaying/format/">Moment.js formatting</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">Example:</p>
<div dir="auto"price': {'fmt': '0.00'},
'url': {'fmt': {'link': True}},
'description': {'fmt': {'truncate': 50}},
'created_date': {'fmt': 'YYYY-MM-DD'}
}"><tt><span>column_formats</span><span>=</span>{<br> <span>'price'</span>: {<span>'fmt'</span>: <span>'0.00'</span>},<br> <span>'url'</span>: {<span>'fmt'</span>: {<span>'link'</span>: <span>True</span>}},<br> <span>'description'</span>: {<span>'fmt'</span>: {<span>'truncate'</span>: <span>50</span>}},<br> <span>'created_date'</span>: {<span>'fmt'</span>: <span>'YYYY-MM-DD'</span>}<br>}</tt></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">sort</h4></div>
<p dir="auto">List of tuples which sort your dataframe. Each tuple contains the column name and sort direction (<code>'ASC'</code> or <code>'DESC'</code>).</p>
<p dir="auto">Example:</p>
<div dir="auto"date', 'DESC'), ('name', 'ASC')]"><tt><span>sort</span><span>=</span>[(<span>'date'</span>, <span>'DESC'</span>), (<span>'name'</span>, <span>'ASC'</span>)]</tt></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">background_mode</h4></div>
<p dir="auto">A string denoting one of the many background displays available in D-Tale.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="left">Value</th>
<th align="left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><code>heatmap-all</code></td>
<td align="left">Turn on heatmap for all numeric columns where colors are determined by the range of values over all numeric columns combined.</td>
</tr>
<tr>
<td align="left"><code>heatmap-col</code></td>
<td align="left">Turn on heatmap for all numeric columns where colors are determined by the range of values in each column.</td>
</tr>
<tr>
<td align="left"><code>heatmap-col-[column name]</code></td>
<td align="left">Turn on heatmap highlighting for a specific column (e.g., <code>heatmap-col-price</code>).</td>
</tr>
<tr>
<td align="left"><code>dtypes</code></td>
<td align="left">Highlight columns based on their data type.</td>
</tr>
<tr>
<td align="left"><code>missing</code></td>
<td align="left">Highlight any missing values (<code>np.nan</code>, empty strings, strings of all spaces).</td>
</tr>
<tr>
<td align="left"><code>outliers</code></td>
<td align="left">Highlight any outliers.</td>
</tr>
<tr>
<td align="left"><code>range</code></td>
<td align="left">Highlight values for any matchers entered in the <code>range_highlights</code> option.</td>
</tr>
<tr>
<td align="left"><code>lowVariance</code></td>
<td align="left">Highlight values with a low variance.</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">range_highlights</h4></div>
<p dir="auto">Dictionary of column name keys and range configurations. If the value for that column matches a condition, it will be shaded with the specified color.</p>
<p dir="auto">Example:</p>
<div dir="auto"a': {
'active': True,
'equals': {'active': True, 'value': 3, 'color': {'r': 255, 'g': 245, 'b': 157, 'a': 1}}, # light yellow
'greaterThan': {'active': True, 'value': 3, 'color': {'r': 80, 'g': 227, 'b': 194, 'a': 1}}, # mint green
'lessThan': {'active': True, 'value': 3, 'color': {'r': 245, 'g': 166, 'b': 35, 'a': 1}}, # orange
}
}"><tt><span>range_highlights</span><span>=</span>{<br> <span>'a'</span>: {<br> <span>'active'</span>: <span>True</span>,<br> <span>'equals'</span>: {<span>'active'</span>: <span>True</span>, <span>'value'</span>: <span>3</span>, <span>'color'</span>: {<span>'r'</span>: <span>255</span>, <span>'g'</span>: <span>245</span>, <span>'b'</span>: <span>157</span>, <span>'a'</span>: <span>1</span>}}, <span># light yellow</span><br> <span>'greaterThan'</span>: {<span>'active'</span>: <span>True</span>, <span>'value'</span>: <span>3</span>, <span>'color'</span>: {<span>'r'</span>: <span>80</span>, <span>'g'</span>: <span>227</span>, <span>'b'</span>: <span>194</span>, <span>'a'</span>: <span>1</span>}}, <span># mint green</span><br> <span>'lessThan'</span>: {<span>'active'</span>: <span>True</span>, <span>'value'</span>: <span>3</span>, <span>'color'</span>: {<span>'r'</span>: <span>245</span>, <span>'g'</span>: <span>166</span>, <span>'b'</span>: <span>35</span>, <span>'a'</span>: <span>1</span>}}, <span># orange</span><br> }<br>}</tt></div>
<p dir="auto">Each column can have the following range conditions:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="left">Condition</th>
<th align="left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><code>equals</code></td>
<td align="left">Highlight cells where the value equals the specified value</td>
</tr>
<tr>
<td align="left"><code>greaterThan</code></td>
<td align="left">Highlight cells where the value is greater than the specified value</td>
</tr>
<tr>
<td align="left"><code>lessThan</code></td>
<td align="left">Highlight cells where the value is less than the specified value</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">Each condition has these properties:</p>
<ul dir="auto">
<li><code>active</code>: Whether this condition is enabled (<code>True</code>/<code>False</code>)</li>
<li><code>value</code>: The value to compare against</li>
<li><code>color</code>: RGBA color object with keys <code>r</code>, <code>g</code>, <code>b</code>, <code>a</code> (values 0-255 for RGB, 0-1 for alpha)</li>
</ul>
<div dir="auto"><h4 tabindex="-1" dir="auto">vertical_headers</h4></div>
<p dir="auto">If set to <code>True</code> then the headers in your grid will be rotated 90 degrees vertically to conserve width.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/vertical_headers.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">column_edit_options</h4></div>
<p dir="auto">Dictionary of column name keys and lists of allowed values. When editing a cell in the specified column, users will be presented with a dropdown of the allowed values instead of free-form text input.</p>
<p dir="auto">Example:</p>
<div dir="auto"status': ['pending', 'approved', 'rejected'],
'priority': ['low', 'medium', 'high', 'critical']
}"><tt><span>column_edit_options</span><span>=</span>{<br> <span>'status'</span>: [<span>'pending'</span>, <span>'approved'</span>, <span>'rejected'</span>],<br> <span>'priority'</span>: [<span>'low'</span>, <span>'medium'</span>, <span>'high'</span>, <span>'critical'</span>]<br>}</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Predefined Filters</h3></div>
<p dir="auto">Users can build their own custom filters which can be used from the front-end using the following code snippet:</p>
<div dir="auto"A', 'B', 'C', 'D', 'E', 'F']
)
dtale.show(df)"><tt><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><span>import</span> <span>dtale</span><br><span>import</span> <span>dtale</span>.<span>predefined_filters</span> <span>as</span> <span>predefined_filters</span><br><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><br><span>global_state</span>.<span>set_app_settings</span>(<span>dict</span>(<span>open_predefined_filters_on_startup</span><span>=</span><span>True</span>))<br><br><span>predefined_filters</span>.<span>set_filters</span>([<br> {<br> <span>"name"</span>: <span>"A and B > 2"</span>,<br> <span>"column"</span>: <span>"A"</span>,<br> <span>"description"</span>: <span>"Filter A with B greater than 2"</span>,<br> <span>"handler"</span>: <span>lambda</span> <span>df</span>, <span>val</span>: <span>df</span>[(<span>df</span>[<span>"A"</span>] <span>==</span> <span>val</span>) <span>&</span> (<span>df</span>[<span>"B"</span>] <span>></span> <span>2</span>)],<br> <span>"input_type"</span>: <span>"input"</span>,<br> <span>"default"</span>: <span>1</span>,<br> <span>"active"</span>: <span>False</span>,<br> },<br> {<br> <span>"name"</span>: <span>"A and (B % 2) == 0"</span>,<br> <span>"column"</span>: <span>"A"</span>,<br> <span>"description"</span>: <span>"Filter A with B mod 2 equals zero (is even)"</span>,<br> <span>"handler"</span>: <span>lambda</span> <span>df</span>, <span>val</span>: <span>df</span>[(<span>df</span>[<span>"A"</span>] <span>==</span> <span>val</span>) <span>&</span> (<span>df</span>[<span>"B"</span>] <span>%</span> <span>2</span> <span>==</span> <span>0</span>)],<br> <span>"input_type"</span>: <span>"select"</span>,<br> <span>"default"</span>: <span>1</span>,<br> <span>"active"</span>: <span>False</span>,<br> },<br> {<br> <span>"name"</span>: <span>"A in values and (B % 2) == 0"</span>,<br> <span>"column"</span>: <span>"A"</span>,<br> <span>"description"</span>: <span>"A is within a group of values and B mod 2 equals zero (is even)"</span>,<br> <span>"handler"</span>: <span>lambda</span> <span>df</span>, <span>val</span>: <span>df</span>[<span>df</span>[<span>"A"</span>].<span>isin</span>(<span>val</span>) <span>&</span> (<span>df</span>[<span>"B"</span>] <span>%</span> <span>2</span> <span>==</span> <span>0</span>)],<br> <span>"input_type"</span>: <span>"multiselect"</span>,<br> <span>"default"</span>: [<span>1</span>],<br> <span>"active"</span>: <span>True</span>,<br> }<br>])<br><br><span>df</span> <span>=</span> <span>pd</span>.<span>DataFrame</span>(<br> ([[<span>1</span>, <span>2</span>, <span>3</span>, <span>4</span>, <span>5</span>, <span>6</span>], [<span>7</span>, <span>8</span>, <span>9</span>, <span>10</span>, <span>11</span>, <span>12</span>], [<span>13</span>, <span>14</span>, <span>15</span>, <span>16</span>, <span>17</span>, <span>18</span>]]),<br> <span>columns</span><span>=</span>[<span>'A'</span>, <span>'B'</span>, <span>'C'</span>, <span>'D'</span>, <span>'E'</span>, <span>'F'</span>]<br>)<br><span>dtale</span>.<span>show</span>(<span>df</span>)</tt></div>
<p dir="auto">This code illustrates the types of inputs you can have on the front end:</p>
<ul dir="auto">
<li><strong>input</strong>: just a simple text input box which users can enter any value they want (if the value specified for <code>"column"</code> is an int or float it will try to convert the string to that data type) and it will be passed to the handler</li>
<li><strong>select</strong>: this creates a dropdown populated with the unique values of <code>"column"</code> (an asynchronous dropdown if the column has a large amount of unique values)</li>
<li><strong>multiselect</strong>: same as "select" but it will allow you to choose multiple values (handy if you want to perform an <code>isin</code> operation in your filter)</li>
</ul>
<p dir="auto">Here is a demo of the functionality:
<a href="/www.youtube.com/watch?v=8mryVxpxjM4"></a></p>
<p dir="auto">If there are any new types of inputs you would like available please don't hesitate to submit a request on the "Issues" page of the repo.</p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Using Swifter</h3></div>
<p dir="auto">Swifter is a package which will increase performance on any <code>apply()</code> function on a pandas series or dataframe. If install the package in your virtual environment</p>
<div dir="auto"><tt>pip install swifter<br><span><span>#</span> or</span><br>pip install dtale[swifter]</tt></div>
<p dir="auto">It will be used for the following operations:</p>
<ul dir="auto">
<li>Standard dataframe formatting in the main grid & chart display</li>
<li>Column Builders
<ul dir="auto">
<li>Type Conversions
<ul dir="auto">
<li>string hex -> int or float</li>
<li>int or float -> hex</li>
<li>mixed -> boolean</li>
<li>int -> timestamp</li>
<li>date -> int</li>
</ul>
</li>
<li>Similarity Distance Calculation</li>
</ul>
</li>
<li>Handling of empty strings when calculating missing counts</li>
<li>Building unique values by data type in "Describe" popup</li>
</ul>
<div dir="auto"><h3 tabindex="-1" dir="auto">Behavior for Wide Dataframes</h3></div>
<p dir="auto">There is currently a performance bottleneck on the front-end when loading "wide dataframes" (dataframes with many columns). The current solution to this problem is that upon initial load of these dataframes to D-Tale any column with an index greater than 100 (going from left to right) will be hidden on the front-end. You can still unhide these columns the same way you would any other and you still have the option to show all columns using the "Describe" popup. Here's a sample of this behavior:</p>
<p dir="auto">Say you loaded this dataframe into D-Tale.</p>
<div dir="auto"col{}'.format(i): list(range(1000)) for i in range(105)}
))"><tt><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><span>import</span> <span>dtale</span><br><br><span>dtale</span>.<span>show</span>(<span>pd</span>.<span>DataFrame</span>(<br> {<span>'col{}'</span>.<span>format</span>(<span>i</span>): <span>list</span>(<span>range</span>(<span>1000</span>)) <span>for</span> <span>i</span> <span>in</span> <span>range</span>(<span>105</span>)}<br>))</tt></div>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/wide_dataframes/initial_load.png"></a></p>
<p dir="auto">You will now have access to a new "Jump To Column" menu item.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/wide_dataframes/jump_to_col_menu_item.png"></a></p>
<p dir="auto">It would be too hard to scroll to the column you're looking for. So now you'll be able to type in the name of the column you're looking for and select it.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/wide_dataframes/jump_to_col_popup.png"></a></p>
<p dir="auto">And now you'll see only the columns you've had locked (we've locked no columns in this example) and the column you chose to jump to.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/wide_dataframes/jump_to_col_done.png"></a></p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Accessing CLI Loaders in Notebook or Console</h3></div>
<p dir="auto">I am pleased to announce that all CLI loaders will be available within notebooks & consoles. Here are some examples (the last working if you've installed <code>dtale[arctic]</code>):</p>
<ul dir="auto">
<li><code>dtale.show_csv(path='test.csv', parse_dates=['date'])</code></li>
<li><code>dtale.show_csv(path='http://csv-endpoint', index_col=0)</code></li>
<li><code>dtale.show_excel(path='test.xlsx', parse_dates=['date'])</code></li>
<li><code>dtale.show_excel(path='test.xls', sheet=)</code></li>
<li><code>dtale.show_excel(path='http://excel-endpoint', index_col=0)</code></li>
<li><code>dtale.show_json(path='http://json-endpoint', parse_dates=['date'])</code></li>
<li><code>dtale.show_json(path='test.json', parse_dates=['date'])</code></li>
<li><code>dtale.show_r(path='text.rda')</code></li>
<li><code>dtale.show_arctic(uri='uri', library='library', symbol='symbol', start='20200101', end='20200101')</code></li>
<li><code>dtale.show_arcticdb(uri='uri', library='library', symbol='symbol', start='20200101', end='20200101', use_store=True)</code></li>
</ul>
<div dir="auto"><h3 tabindex="-1" dir="auto">Using ArcticDB as your data store for D-Tale</h3></div>
<p dir="auto">So one of the major drawbacks of using D-Tale is that it stores a copy of your dataframe in memory (unless you specify the <code>inplace=True</code> when calling <code>dtale.show</code>). One way around this is to switch your storage mechanism to ArcticDB. This will use ArcticDB's <a href="/docs.arcticdb.io/api/query_builder">QueryBuilder</a> to perform all data loading and filtering. This will significantly drop your memory footprint, but it will remove a lot of the original D-Tale functionality:</p>
<ul dir="auto">
<li>Custom Filtering</li>
<li>Range filtering in Numeric Column Filters</li>
<li>Regex filtering on String Column Filters</li>
<li>Editing Cells</li>
<li>Data Reshaping</li>
<li>Dataframe Functions</li>
<li>Drop Filtered Rows</li>
<li>Sorting</li>
</ul>
<p dir="auto">If the symbol you're loading from ArcticDB contains more than 1,000,000 rows then you will also lose the following:</p>
<ul dir="auto">
<li>Column Filtering using dropdowns of unique values (you'll have to manually type your values)</li>
<li>Outlier Highlighting</li>
<li>Most of the details in the "Describe" screen</li>
</ul>
<p dir="auto">In order to update your storage mechanism there are a few options, the first being <code>use_arcticdb_store</code>:</p>
<div dir="auto"lmdb:///<path>')
dtale.show_arcticdb(library='my_lib', symbol='my_symbol')"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>import</span> <span>dtale</span><br><br><span>global_state</span>.<span>use_arcticdb_store</span>(<span>uri</span><span>=</span><span>'lmdb:///<path>'</span>)<br><span>dtale</span>.<span>show_arcticdb</span>(<span>library</span><span>=</span><span>'my_lib'</span>, <span>symbol</span><span>=</span><span>'my_symbol'</span>)</tt></div>
<p dir="auto">Or you can set your library ahead of time so you can use <code>dtale.show</code>:</p>
<div dir="auto"lmdb:///<path>', library='my_lib')
dtale.show('my_symbol')"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>import</span> <span>dtale</span><br><br><span>global_state</span>.<span>use_arcticdb_store</span>(<span>uri</span><span>=</span><span>'lmdb:///<path>'</span>, <span>library</span><span>=</span><span>'my_lib'</span>)<br><span>dtale</span>.<span>show</span>(<span>'my_symbol'</span>)</tt></div>
<p dir="auto">Or you can set your library using <code>dtale.show</code> with a pipe-delimited identifier:</p>
<div dir="auto"lmdb:///<path>')
dtale.show('my_lib|my_symbol')"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>import</span> <span>dtale</span><br><br><span>global_state</span>.<span>use_arcticdb_store</span>(<span>uri</span><span>=</span><span>'lmdb:///<path>'</span>)<br><span>dtale</span>.<span>show</span>(<span>'my_lib|my_symbol'</span>)</tt></div>
<p dir="auto">You can also do everything using <code>dtale.show_arcticdb</code>:</p>
<div dir="auto"lmdb:///<path>', library='my_lib', symbol='my_symbol', use_store=True)"><tt><span>import</span> <span>dtale</span><br><br><span>dtale</span>.<span>show_arcticdb</span>(<span>uri</span><span>=</span><span>'lmdb:///<path>'</span>, <span>library</span><span>=</span><span>'my_lib'</span>, <span>symbol</span><span>=</span><span>'my_symbol'</span>, <span>use_store</span><span>=</span><span>True</span>)</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Navigating to different libraries/symbols in your ArcticDB database</h3></div>
<p dir="auto">When starting D-Tale with no data</p>
<div dir="auto"lmdb:///<path>')
dtale.show()"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>import</span> <span>dtale</span><br><br><span>global_state</span>.<span>use_arcticdb_store</span>(<span>uri</span><span>=</span><span>'lmdb:///<path>'</span>)<br><span>dtale</span>.<span>show</span>()</tt></div>
<p dir="auto">you'll be presented with this screen on startup
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/arcticdb/select_library_and_symbol.png"></a></p>
<p dir="auto">Once you choose a library and a symbol you can click "Load" and it will bring you to the main grid comprised of the data for that symbol.</p>
<p dir="auto">You can also view information about the symbol you've selected before loading it by clicking the "Info" button
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/arcticdb/description.png"></a></p>
<div dir="auto"><h2 tabindex="-1" dir="auto">UI</h2></div>
<p dir="auto">Once you have kicked off your D-Tale session please copy & paste the link on the last line of output in your browser
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Browser1.png"></a></p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Dimensions/Ribbon Menu/Main Menu</h3></div>
<p dir="auto">The information in the upper right-hand corner gives grid dimensions <a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Info_cell.png"></a></p>
<ul dir="auto">
<li>lower-left => row count</li>
<li>upper-right => column count</li>
</ul>
<p dir="auto">Ribbon Menu</p>
<ul dir="auto">
<li>hovering around to top of your browser will display a menu items (similar to the ones in the main menu) across the top of the screen</li>
<li>to close the menu simply click outside the menu and/or dropdowns from the menu</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Ribbon_menu.png"></a></p>
<p dir="auto">Main Menu</p>
<ul dir="auto">
<li>clicking the triangle displays the menu of standard functions (click outside menu to close it)</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Info_menu_small.png"></a></p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Header</h3></div>
<p dir="auto">The header gives users an idea of what operations have taken place on your data (sorts, filters, hidden columns). These values will be persisted across broswer instances. So if you perform one of these operations and then send a link to one of your colleagues they will see the same thing :)</p>
<p dir="auto">Notice the "X" icon on the right of each display. Clicking this will remove those operations.</p>
<p dir="auto">When performing multiple of the same operation the description will become too large to display so the display will truncate the description and if users click it they will be presented with a tooltip where you can crop individual operations. Here are some examples:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Sorts</th>
<th>Filters</th>
<th>Hidden Columns</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/header/sorts.PNG"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/header/filters.PNG"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/header/hidden.PNG"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h3 tabindex="-1" dir="auto">Resize Columns</h3></div>
<p dir="auto">Currently there are two ways which you can resize columns.</p>
<ul dir="auto">
<li>Dragging the right border of the column's header cell.</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/gifs/resize_columns_w_drag.gif"></a></p>
<ul dir="auto">
<li>Altering the "Maximum Column Width" property from the ribbon menu.</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/gifs/resize_columns_max_width.gif"></a></p>
<ul dir="auto">
<li><strong>Side Note:</strong> You can also set the <code>max_column_width</code> property ahead of time in your <a href="/github.com/man-group/dtale/blob/master/docs/CONFIGURATION.md">global configuration</a> or programmatically using:</li>
</ul>
<div dir="auto"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><br><span>global_state</span>.<span>set_app_settings</span>(<span>dict</span>(<span>max_column_width</span><span>=</span><span>100</span>))</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Editing Cells</h3></div>
<p dir="auto">You may edit any cells in your grid (with the exception of the row indexes or headers, the ladder can be edited using the Rename column menu function).</p>
<p dir="auto">In order to edit a cell simply double-click on it. This will convert it into a text-input field and you should see a blinking cursor. In addition to turning that cell into an input it will also display an input at the top of the screen for better viewing of long strings. It is assumed that the value you type in will match the data type of the column you editing. For example:</p>
<ul dir="auto">
<li>integers -> should be a valid positive or negative integer</li>
<li>float -> should be a valid positive or negative float</li>
<li>string -> any valid string will do</li>
<li>category -> either a pre-existing category or this will create a new category for (so beware!)</li>
<li>date, timestamp, timedelta -> should be valid string versions of each</li>
<li>boolean -> any string you input will be converted to lowercase and if it equals "true" then it will make the cell <code>True</code>, otherwise <code>False</code></li>
</ul>
<p dir="auto">Users can make use of two protected values as well:</p>
<ul dir="auto">
<li>"nan" -> <code>numpy.nan</code></li>
<li>"inf" -> <code>numpy.inf</code></li>
</ul>
<p dir="auto">To save your change simply press "Enter" or to cancel your changes press "Esc".</p>
<p dir="auto">If there is a conversion issue with the value you have entered it will display a popup with the specific exception in question.</p>
<p dir="auto">Here's a quick demo:</p>
<p dir="auto"><a href="/www.youtube.com/watch?v=MY5w0m_4IAc"></a></p>
<p dir="auto">Here's a demo of editing cells with long strings:</p>
<p dir="auto"><a href="/www.youtube.com/watch?v=3p9ltzdBaDQ"></a></p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Copy Cells Into Clipboard</h3></div>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Select</th>
<th align="center">Copy</th>
<th align="center">Paste</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/select_range1.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/select_range2.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/select_range3.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">One request that I have heard time and time again while working on D-Tale is "it would be great to be able to copy a range of cells into excel". Well here is how that is accomplished:</p>
<ol dir="auto">
<li>Shift + Click on a cell</li>
<li>Shift + Click on another cell (this will trigger a popup)</li>
<li>Choose whether you want to include headers in your copy by clicking the checkbox</li>
<li>Click Yes</li>
<li>Go to your excel workbook and execute Ctrl + V or manually choose "Paste"</li>
</ol>
<ul dir="auto">
<li>You can also paste this into a standard text editor and what you're left with is tab-delimited data</li>
</ul>
<div dir="auto"><h3 tabindex="-1" dir="auto">Main Menu Functions</h3></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">XArray Operations</h4></div>
<ul dir="auto">
<li><strong>Convert To XArray</strong>: If you have are currently viewing a pandas dataframe in D-Tale you will be given this option to convert your data to an <code>xarray.Dataset</code>. It is as simple as selecting one or many columns as an index and then your dataframe will be converted to a dataset (<code>df.set_index([...]).to_xarray()</code>) which makes toggling between indexes slices much easier.</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/xarray_indexes.png"></a></p>
<ul dir="auto">
<li><strong>XArray Dimensions</strong>: If you are currently viewing data associated with an <code>xarray.Dataset</code> you will be given the ability to toggle which dimension coordinates you're viewing by clicking this button. You can select values for all, some or none (all data - no filter) of your coordinates and the data displayed in your grid will match your selections. Under the hood the code being executed is as follows: <code>ds.sel(dim1=coord1,...).to_dataframe()</code></li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/xarray_dimensions.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Describe</h4></div>
<p dir="auto">View all the columns & their data types as well as individual details of each column</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Describe.png"></a></p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Data Type</th>
<th align="center">Display</th>
<th align="center">Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td>date</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Describe_date.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td>string</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Describe_string.png"></a></td>
<td align="center">If you have less than or equal to 100 unique values they will be displayed at the bottom of your popup</td>
</tr>
<tr>
<td>int</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Describe_int.png"></a></td>
<td align="center">Anything with standard numeric classifications (min, max, 25%, 50%, 75%) will have a nice boxplot with the mean (if it exists) displayed as an outlier if you look closely.</td>
</tr>
<tr>
<td>float</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Describe_float.png"></a></td>
<td align="center"></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Outlier Detection</h4></div>
<p dir="auto">When viewing integer & float columns in the "Describe" popup you will see in the lower right-hand corner a toggle for Uniques & Outliers.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Outliers</th>
<th>Filter</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/outliers.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/outlier_filter.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">If you click the "Outliers" toggle this will load the top 100 outliers in your column based on the following code snippet:</p>
<div dir="auto"><tt><span>s</span> <span>=</span> <span>df</span>[<span>column</span>]<br><span>q1</span> <span>=</span> <span>s</span>.<span>quantile</span>(<span>0.25</span>)<br><span>q3</span> <span>=</span> <span>s</span>.<span>quantile</span>(<span>0.75</span>)<br><span>iqr</span> <span>=</span> <span>q3</span> <span>-</span> <span>q1</span><br><span>iqr_lower</span> <span>=</span> <span>q1</span> <span>-</span> <span>1.5</span> <span>*</span> <span>iqr</span><br><span>iqr_upper</span> <span>=</span> <span>q3</span> <span>+</span> <span>1.5</span> <span>*</span> <span>iqr</span><br><span>outliers</span> <span>=</span> <span>s</span>[(<span>s</span> <span><</span> <span>iqr_lower</span>) <span>|</span> (<span>s</span> <span>></span> <span>iqr_upper</span>)]</tt></div>
<p dir="auto">If you click on the "Apply outlier filter" link this will add an addtional "outlier" filter for this column which can be removed from the header or the custom filter shown in picture above to the right.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Custom Filter</h4></div>
<p dir="auto"><strong>Starting with version 3.7.0 this feature will be turned off by default.
Custom filters are vulnerable to code injection attacks, please only use in trusted environments.</strong></p>
<p dir="auto"><strong>You can turn this feature on by doing one of the following:</strong></p>
<ul dir="auto">
<li><strong>add <code>enable_custom_filters=True</code> to your <code>dtale.show</code> call</strong></li>
<li><strong>add <code>enable_custom_filters = True</code> to the [app] section of your dtale.ini config file (<a href="/github.com/man-group/dtale/blob/master/docs/CONFIGURATION.md">more info</a>)</strong></li>
<li><strong>run this code before calling dtale.show:</strong></li>
</ul>
<div dir="auto"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>global_state</span>.<span>set_app_settings</span>(<span>dict</span>(<span>enable_custom_filters</span><span>=</span><span>True</span>))</tt></div>
<p dir="auto">Apply a custom pandas <code>query</code> to your data (link to pandas documentation included in popup)</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Editing</th>
<th align="center">Result</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Filter_apply.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Post_filter.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">You can also see any outlier or column filters you've applied (which will be included in addition to your custom query) and remove them if you'd like.</p>
<p dir="auto">Context Variables are user-defined values passed in via the <code>context_variables</code> argument to dtale.show(); they can be referenced in filters by prefixing the variable name with '@'.</p>
<p dir="auto">For example, here is how you can use context variables in a pandas query:</p>
<div dir="auto"Joe', age=7),
dict(name='Bob', age=23),
dict(name='Ann', age=45),
dict(name='Cat', age=88),
])
two_oldest_ages = df['age'].nlargest(2)
df.query('age in @two_oldest_ages')"><tt><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><br><span>df</span> <span>=</span> <span>pd</span>.<span>DataFrame</span>([<br> <span>dict</span>(<span>name</span><span>=</span><span>'Joe'</span>, <span>age</span><span>=</span><span>7</span>),<br> <span>dict</span>(<span>name</span><span>=</span><span>'Bob'</span>, <span>age</span><span>=</span><span>23</span>),<br> <span>dict</span>(<span>name</span><span>=</span><span>'Ann'</span>, <span>age</span><span>=</span><span>45</span>),<br> <span>dict</span>(<span>name</span><span>=</span><span>'Cat'</span>, <span>age</span><span>=</span><span>88</span>),<br>])<br><span>two_oldest_ages</span> <span>=</span> <span>df</span>[<span>'age'</span>].<span>nlargest</span>(<span>2</span>)<br><span>df</span>.<span>query</span>(<span>'age in @two_oldest_ages'</span>)</tt></div>
<p dir="auto">And here is how you would pass that context variable to D-Tale: <code>dtale.show(df, context_variables=dict(two_oldest_ages=two_oldest_ages))</code></p>
<p dir="auto">Here's some nice documentation on the performance of <a href="/pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html#pandas-eval-performance">pandas queries</a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Dataframe Functions</h4></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=G6wNS9-lG04"></a></p>
<p dir="auto">This video shows you how to build the following:</p>
<ul dir="auto">
<li>Numeric: adding/subtracting two columns or columns with static values</li>
<li>Bins: bucketing values using pandas cut & qcut as well as assigning custom labels</li>
<li>Dates: retrieving date properties (hour, weekday, month...) as well as conversions (month end)</li>
<li>Random: columns of data type (int, float, string & date) populated with random uniformly distributed values.</li>
<li>Type Conversion: switch columns from one data type to another, fun. </li>
</ul>
<div dir="auto"><h4 tabindex="-1" dir="auto">Merge & Stack</h4></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=ignDS6OaGVQ"></a></p>
<p dir="auto">This feature allows users to merge or stack (vertically concatenate) dataframes they have loaded into D-Tale. They can also upload additional data to D-Tale while wihin this feature. The demo shown above goes over the following actions:</p>
<ul dir="auto">
<li>Editing of parameters to either a pandas merge or stack (vertical concatenation) of dataframes
<ul dir="auto">
<li>Viewing direct examples of each from the pandas documentation</li>
</ul>
</li>
<li>Selection of dataframes</li>
<li>Uploading of additional dataframes from an excel file</li>
<li>Viewing code & resulting data from a merge or stack</li>
</ul>
<div dir="auto"><h4 tabindex="-1" dir="auto">Summarize Data</h4></div>
<p dir="auto">This is very powerful functionality which allows users to create a new data from currently loaded data. The operations currently available are:</p>
<ul dir="auto">
<li><strong>Aggregation</strong>: consolidate data by running different aggregations on columns by a specific index</li>
<li><strong>Pivot</strong>: this is simple wrapper around <a href="/pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.pivot.html">pandas.Dataframe.pivot</a> and <a href="/pandas.pydata.org/pandas-docs/stable/reference/api/pandas.pivot_table.html">pandas.pivot_table</a></li>
<li><strong>Transpose</strong>: transpose your data on a index (be careful dataframes can get very wide if your index has many unique values)</li>
</ul>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Function</th>
<th align="center">Data</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">No Reshaping</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/original_data.png"></a></td>
</tr>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/agg_col.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/agg_col_data.png"></a></td>
</tr>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/agg_func.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/agg_func_data.png"></a></td>
</tr>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/pivot.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/pivot_data.png"></a></td>
</tr>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/transpose.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/reshape/transpose_data.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto"><a href="/www.youtube.com/watch?v=fYsxogXKZ2c"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Duplicates</h4></div>
<p dir="auto">Remove duplicate columns/values from your data as well as extract duplicates out into separate instances.</p>
<p dir="auto">The folowing screen shots are for a dataframe with the following data:
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/duplicates/data.png"></a></p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Function</th>
<th align="center">Description</th>
<th align="center">Preview</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><strong>Remove Duplicate Columns</strong></td>
<td align="center">Remove any columns that contain the same data as another and you can either keep the first, last or none of these columns that match this criteria. You can test which columns will be removed by clicking the "View Duplicates" button.</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/duplicates/columns.png"></a></td>
</tr>
<tr>
<td align="center"><strong>Remove Duplicate Column Names</strong></td>
<td align="center">Remove any columns with the same name (name comparison is case-insensitive) and you can either keep the first, last or none of these columns that match this criteria. You can test which columns will be removed by clicking the "View Duplicates" button.</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/duplicates/column_names.png"></a></td>
</tr>
<tr>
<td align="center"><strong>Remove Duplicate Rows</strong></td>
<td align="center">Remove any rows from your dataframe where the values of a subset of columns are considered duplicates. You can choose to keep the first, last or none of the rows considered duplicated.</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/duplicates/rows.png"></a></td>
</tr>
<tr>
<td align="center"><strong>Show Duplicates</strong></td>
<td align="center">Break any duplicate rows (based on a subset of columns) out into another dataframe viewable in your D-Tale session. You can choose to view all duplicates or select specific groups based on the duplicated value.</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/duplicates/show.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Missing Analysis</h4></div>
<p dir="auto">Display charts analyzing the presence of missing (NaN) data in your dataset using the <a href="/github.com/ResidentMario/missingno">missingno</a> pacakage. You can also open them in a tab by themselves or export them to a static PNG using the links in the upper right corner. You can also close the side panel using the ESC key.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Chart</th>
<th>Sample</th>
</tr>
</thead>
<tbody>
<tr>
<td>Matrix</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/missingno/matrix.png"></a></td>
</tr>
<tr>
<td>Bar</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/missingno/bar.png"></a></td>
</tr>
<tr>
<td>Heatmap</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/missingno/heatmap.png"></a></td>
</tr>
<tr>
<td>Dendrogram</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/missingno/dendrogram.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Charts</h4></div>
<p dir="auto">Build custom charts based off your data(powered by <a href="/github.com/plotly/dash">plotly/dash</a>).</p>
<ul dir="auto">
<li>The Charts will open in a tab because of the fact there is so much functionality offered there you'll probably want to be able to reference the main grid data in the original tab</li>
<li>To build a chart you must pick a value for X & Y inputs which effectively drive what data is along the X & Y axes
<ul dir="auto">
<li>If you are working with a 3-Dimensional chart (heatmap, 3D Scatter, Surface) you'll need to enter a value for the Z axis as well</li>
</ul>
</li>
<li>Once you have entered all the required axes a chart will be built</li>
<li>If your data along the x-axis (or combination of x & y in the case of 3D charts) has duplicates you have three options:
<ul dir="auto">
<li>Specify a group, which will create series for each group</li>
<li>Specify an aggregation, you can choose from one of the following: Count, First, Last, Mean, Median, Minimum, Maximum, Standard Deviation, Variance, Mean Absolute Deviation, Product of All Items, Sum, Rolling
<ul dir="auto">
<li>Specifying a "Rolling" aggregation will also require a Window & a Computation (Correlation, Count, Covariance, Kurtosis, Maximum, Mean, Median, Minimum, Skew, Standard Deviation, Sum or Variance)</li>
<li>For heatmaps you will also have access to the "Correlation" aggregation since viewing correlation matrices in heatmaps is very useful. This aggregation is not supported elsewhere</li>
</ul>
</li>
<li>Specify both a group & an aggregation</li>
</ul>
</li>
<li>You now have the ability to toggle between different chart types: line, bar, pie, wordcloud, heatmap, 3D scatter & surface</li>
<li>If you have specified a group then you have the ability between showing all series in one chart and breaking each series out into its own chart "Chart per Group"</li>
</ul>
<p dir="auto">Here are some examples:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Chart Type</th>
<th align="center">Chart</th>
<th align="center">Chart per Group</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">line</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/line.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/line_pg.png"></a></td>
</tr>
<tr>
<td align="center">bar</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/bar.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/bar_pg.png"></a></td>
</tr>
<tr>
<td align="center">stacked</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/stacked.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/stacked_pg.png"></a></td>
</tr>
<tr>
<td align="center">pie</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/pie.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td align="center">wordcloud</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/wordcloud.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/wordcloud_pg.png"></a></td>
</tr>
<tr>
<td align="center">heatmap</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/heatmap.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td align="center">3D scatter</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/3d_scatter.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td align="center">surface</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/surface.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td align="center">Maps (Scatter GEO)</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/scattergeo.png"></a></td>
<td align="center"></td>
</tr>
<tr>
<td align="center">Maps (Choropleth)</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/choropleth.png"></a></td>
<td align="center"></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">Y-Axis Toggling</p>
<p dir="auto">Users now have the ability to toggle between 3 different behaviors for their y-axis display:</p>
<ul dir="auto">
<li><em>Default</em>: selecting this option will use the default behavior that comes with plotly for your chart's y-axis</li>
<li><em>Single</em>: this allows users to set the range of all series in your chart to be on the same basis as well as making that basis (min/max) editable</li>
<li><em>Multi</em>: this allows users to give each series its own y-axis and making that axis' range editable</li>
</ul>
<p dir="auto">Here's a quick tutorial: <a href="/www.youtube.com/watch?v=asblF-rAACY"></a></p>
<p dir="auto">And some screenshots:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Default</th>
<th align="center">Single</th>
<th align="center">Multi</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/axis_toggle/default.PNG"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/axis_toggle/single.PNG"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/axis_toggle/multi.PNG"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">With a bar chart that only has a single Y-Axis you have the ability to sort the bars based on the values for the Y-Axis</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Pre-sort</th>
<th align="center">Post-sort</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/bar_presort.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/charts/bar_postsort.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto"><strong>Popup Charts</strong></p>
<p dir="auto">Viewing multiple charts at once and want to separate one out into its own window or simply move one off to the side so you can work on building another for comparison? Well now you can by clicking the "Popup" button </p>
<p dir="auto"><strong>Copy Link</strong></p>
<p dir="auto">Want to send what you're looking at to someone else? Simply click the "Copy Link" button and it will save a pre-populated chart URL into your clipboard. As long as your D-Tale process is still running when that link is opened you will see your original chart.</p>
<p dir="auto"><strong>Exporting Charts</strong></p>
<p dir="auto">You can now export your dash charts (with the exception of Wordclouds) to static HTML files which can be emailed to others or saved down to be viewed at a later time. The best part is that all of the javascript for plotly is embedded in these files so the nice zooming, panning, etc is still available! </p>
<p dir="auto"><strong>Exporting CSV</strong></p>
<p dir="auto">I've been asked about being able to export the data that is contained within your chart to a CSV for further analysis in tools like Excel. This button makes that possible.</p>
<div dir="auto"><h3 tabindex="-1" dir="auto">OFFLINE CHARTS</h3></div>
<p dir="auto">Want to run D-Tale in a jupyter notebook and build a chart that will still be displayed even after your D-Tale process has shutdown? Now you can! Here's an example code snippet show how to use it:</p>
<divz{}'.format(j) for j in range(5)])
return pd.concat([df, rand_data], axis=1)
d = dtale.show(test_data())
d.offline_chart(chart_type='bar', x='x', y='z3', agg='sum')"><tt><code>import dtale<br><br>def test_data():<br> import random<br> import pandas as pd<br> import numpy as np<br><br> df = pd.DataFrame([<br> dict(x=i, y=i % 2)<br> for i in range(30)<br> ])<br> rand_data = pd.DataFrame(np.random.randn(len(df), 5), columns=['z{}'.format(j) for j in range(5)])<br> return pd.concat([df, rand_data], axis=1)<br><br>d = dtale.show(test_data())<br>d.offline_chart(chart_type='bar', x='x', y='z3', agg='sum')<br></code></tt></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=DseSmc3fZvc"></a></p>
<p dir="auto"><strong>Pro Tip: If generating offline charts in jupyter notebooks and you run out of memory please add the following to your command-line when starting jupyter</strong></p>
<p dir="auto"><code>--NotebookApp.iopub_data_rate_limit=1.0e10</code></p>
<p dir="auto"><strong>Disclaimer: Long Running Chart Requests</strong></p>
<p dir="auto">If you choose to build a chart that requires a lot of computational resources then it will take some time to run. Based on the way Flask & plotly/dash interact this will block you from performing any other request until it completes. There are two courses of action in this situation:</p>
<ol dir="auto">
<li>Restart your jupyter notebook kernel or python console</li>
<li>Open a new D-Tale session on a different port than the current session. You can do that with the following command: <code>dtale.show(df, port=[any open port], force=True)</code></li>
</ol>
<p dir="auto">If you miss the legacy (non-plotly/dash) charts, not to worry! They are still available from the link in the upper-right corner, but on for a limited time...
Here is the documentation for those: <a href="/github.com/man-group/dtale/blob/master/docs/LEGACY_CHARTS.md">Legacy Charts</a></p>
<p dir="auto"><strong>Your Feedback is Valuable</strong></p>
<p dir="auto">This is a very powerful feature with many more features that could be offered (linked subplots, different statistical aggregations, etc...) so please submit issues :)</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Network Viewer</h4></div>
<p dir="auto">This tool gives users the ability to visualize directed graphs. For the screenshots I'll beshowing for this functionality we'll be working off a dataframe with the following data:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/data.png"></a></p>
<p dir="auto">Start by selecting columns containing the "To" and "From" values for the nodes in you network and then click "Load":
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_to_from.png"></a></p>
<p dir="auto">You can also see instructions on to interact with the network by expanding the directions section by clicking on the header "Network Viewer" at the top. You can also view details about the network provided by the package <a href="/github.com/networkx">networkx</a> by clicking the header "Network Analysis".
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_expanded.png"></a></p>
<p dir="auto">Select a column containing weighting for the edges of the nodes in the "Weight" column and click "Load":
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_weight.png"></a></p>
<p dir="auto">Select a column containing group information for each node in the "From" column by populating "Group" and then clicking "Load":
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_group.png"></a></p>
<p dir="auto">Perform shortest path analysis by doing a Shift+Click on two nodes:
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_shortest_path.png"></a></p>
<p dir="auto">View direct descendants of each node by clicking on it:
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/network/network_descendants.png"></a></p>
<p dir="auto">You can zoom in on nodes by double-clicking and zoom back out by pressing "Esc".</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Correlations</h4></div>
<p dir="auto">Shows a pearson correlation matrix of all numeric columns against all other numeric columns</p>
<ul dir="auto">
<li>By default, it will show a grid of pearson correlations (filtering available by using drop-down see 2nd table of screenshots)</li>
<li>If you have a date-type column, you can click an individual cell and see a timeseries of pearson correlations for that column combination
<ul dir="auto">
<li>Currently if you have multiple date-type columns you will have the ability to toggle between them by way of a drop-down</li>
</ul>
</li>
<li>Furthermore, you can click on individual points in the timeseries to view the scatter plot of the points going into that correlation
<ul dir="auto">
<li>Within the scatter plot section you can also view the details of the PPS for those data points in the chart by hovering over the number next to "PPS"</li>
</ul>
</li>
</ul>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Matrix</th>
<th>PPS</th>
<th>Timeseries</th>
<th>Scatter</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_pps.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_ts.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_scatter.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Col1 Filtered</th>
<th>Col2 Filtered</th>
<th>Col1 & Col2 Filtered</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_col1.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_col2.png"></a></td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Correlations_both.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">When the data being viewed in D-Tale has date or timestamp columns but for each date/timestamp vlaue there is only one row of data the behavior of the Correlations popup is a little different</p>
<ul dir="auto">
<li>Instead of a timeseries correlation chart the user is given a rolling correlation chart which can have the window (default: 10) altered</li>
<li>The scatter chart will be created when a user clicks on a point in the rollign correlation chart. The data displayed in the scatter will be for the ranges of dates involved in the rolling correlation for that date.</li>
</ul>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Data</th>
<th align="center">Correlations</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/rolling_corr_data.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/rolling_corr.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Predictive Power Score</h4></div>
<p dir="auto">Predictive Power Score (using the package <a href="/github.com/8080labs/ppscore">ppscore</a>) is an asymmetric, data-type-agnostic score that can detect linear or non-linear relationships between two columns. The score ranges from 0 (no predictive power) to 1 (perfect predictive power). It can be used as an alternative to the correlation (matrix). WARNING: This could take a while to load.</p>
<p dir="auto">This page works similar to the Correlations page but uses the PPS calcuation to populate the grid and by clicking on cells you can view the details of the PPS for those two columns in question.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/rolling_corr_data.png"></a>|<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/pps.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Heat Map</h4></div>
<p dir="auto">This will hide any non-float or non-int columns (with the exception of the index on the right) and apply a color to the background of each cell.</p>
<ul dir="auto">
<li>Each float is renormalized to be a value between 0 and 1.0</li>
<li>You have two options for the renormalization
<ul dir="auto">
<li><strong>By Col</strong>: each value is calculated based on the min/max of its column</li>
<li><strong>Overall</strong>: each value is caluclated by the overall min/max of all the non-hidden float/int columns in the dataset</li>
</ul>
</li>
<li>Each renormalized value is passed to a color scale of red(0) - yellow(0.5) - green(1.0)
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Heatmap.png"></a></li>
</ul>
<p dir="auto">Turn off Heat Map by clicking menu option you previously selected one more time</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Highlight Dtypes</h4></div>
<p dir="auto">This is a quick way to check and see if your data has been categorized correctly. By clicking this menu option it will assign a specific background color to each column of a specific data type.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">category</th>
<th align="center">timedelta</th>
<th align="center">float</th>
<th align="center">int</th>
<th align="center">date</th>
<th align="center">string</th>
<th align="center">bool</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">purple</td>
<td align="center">orange</td>
<td align="center">green</td>
<td align="center">light blue</td>
<td align="center">pink</td>
<td align="center">white</td>
<td align="center">yellow</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_dtypes.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Highlight Missing</h4></div>
<ul dir="auto">
<li>Any cells which contain <code>nan</code> values will be highlighted in yellow.</li>
<li>Any string column cells which are empty strings or strings consisting only of spaces will be highlighted in orange.</li>
<li>will be prepended to any column header which contains missing values.</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_missing.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Highlight Outliers</h4></div>
<p dir="auto">Highlight any cells for numeric columns which surpass the upper or lower bounds of a custom outlier computation.</p>
<ul dir="auto">
<li>Lower bounds outliers will be on a red scale, where the darker reds will be near the maximum value for the column.</li>
<li>Upper bounds outliers will be on a blue scale, where the darker blues will be closer to the minimum value for the column.</li>
<li> will be prepended to any column header which contains outliers.</li>
</ul>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_outliers.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Highlight Range</h4></div>
<p dir="auto">Highlight any range of numeric cells based on three different criteria:</p>
<ul dir="auto">
<li>equals</li>
<li>greater than</li>
<li>less than</li>
</ul>
<p dir="auto">You can activate as many of these criteria as you'd like nad they will be treated as an "or" expression. For example, <code>(x == 0) or (x < -1) or (x > 1)</code></p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Selections</th>
<th align="center">Output</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_range_selections.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_range_output.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Low Variance Flag</h4></div>
<p dir="auto">Show flags on column headers where both these conditions are true:</p>
<ul dir="auto">
<li>Count of unique values / column size < 10%</li>
<li>Count of most common value / Count of second most common value > 20</li>
</ul>
<p dir="auto">Here's an example of what this will look like when you apply it:
<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/highlight_range_selections.png"></a>|<a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/low_variance.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Code Exports</h4></div>
<p dir="auto"><em>Code Exports</em> are small snippets of code representing the current state of the grid you're viewing including things like:</p>
<ul dir="auto">
<li>columns built</li>
<li>filtering</li>
<li>sorting</li>
</ul>
<p dir="auto">Other code exports available are:</p>
<ul dir="auto">
<li>Describe (Column Analysis)</li>
<li>Correlations (grid, timeseries chart & scatter chart)</li>
<li>Charts built using the Chart Builder</li>
</ul>
<p dir="auto"><a href="/www.youtube.com/watch?v=6CkKgpv3d6I"></a></p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Type</th>
<th align="center">Code Export</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">Main Grid</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/main.png"></a></td>
</tr>
<tr>
<td align="center">Histogram</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/histogram.png"></a></td>
</tr>
<tr>
<td align="center">Describe</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/describe.png"></a></td>
</tr>
<tr>
<td align="center">Correlation Grid</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/main.png"></a></td>
</tr>
<tr>
<td align="center">Correlation Timeseries</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/corr_ts.png"></a></td>
</tr>
<tr>
<td align="center">Correlation Scatter</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/corr_scatter.png"></a></td>
</tr>
<tr>
<td align="center">Charts</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/code_export/charts.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Export CSV</h4></div>
<p dir="auto">Export your current data to either a CSV or TSV file:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/export_csv.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Load Data & Sample Datasets</h4></div>
<p dir="auto">So either when starting D-Tale with no pre-loaded data or after you've already loaded some data you now have the ability to load data or choose from some sample datasets directly from the GUI:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/load_data.png"></a></p>
<p dir="auto">Here's the options at you disposal:</p>
<ul dir="auto">
<li>Load a CSV/TSV file by dragging a file to the dropzone in the top or select a file by clicking the dropzone</li>
<li>Load a CSV/TSV or JSON directly from the web by entering a URL (also throw in a proxy if you are using one)</li>
<li>Choose from one of our sample datasets:
<ul dir="auto">
<li>US COVID-19 data from NY Times (updated daily)</li>
<li>Script breakdowns of popular shows Seinfeld & The Simpsons</li>
<li>Movie dataset containing release date, director, actors, box office, reviews...</li>
<li>Video games and their sales</li>
<li>pandas.util.testing.makeTimeDataFrame</li>
</ul>
</li>
</ul>
<p dir="auto"><strong>Starting with version 3.9.0 web uploads will be turned off by default.
Web uploads are vulnerable to blind server side request forgery, please only use in trusted environments.</strong></p>
<p dir="auto"><strong>You can turn this feature on by doing one of the following:</strong></p>
<ul dir="auto">
<li><strong>add <code>enable_web_uploads=True</code> to your <code>dtale.show</code> call</strong></li>
<li><strong>add <code>enable_web_uploads = False</code> to the [app] section of your dtale.ini config file (<a href="/github.com/man-group/dtale/blob/master/docs/CONFIGURATION.md">more info</a>)</strong></li>
<li><strong>run this code before calling dtale.show:</strong></li>
</ul>
<div dir="auto"><tt><span>import</span> <span>dtale</span>.<span>global_state</span> <span>as</span> <span>global_state</span><br><span>global_state</span>.<span>set_app_settings</span>(<span>dict</span>(<span>enable_web_uploads</span><span>=</span><span>True</span>))</tt></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">Instances</h4></div>
<p dir="auto">This will give you information about other D-Tale instances are running under your current Python process.</p>
<p dir="auto">For example, if you ran the following script:</p>
<div dir="auto"><tt><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><br><span>import</span> <span>dtale</span><br><br><span>dtale</span>.<span>show</span>(<span>pd</span>.<span>DataFrame</span>([<span>dict</span>(<span>foo</span><span>=</span><span>1</span>, <span>bar</span><span>=</span><span>2</span>, <span>biz</span><span>=</span><span>3</span>, <span>baz</span><span>=</span><span>4</span>, <span>snoopy_D_O_double_gizzle</span><span>=</span><span>5</span>)]))<br><span>dtale</span>.<span>show</span>(<span>pd</span>.<span>DataFrame</span>([<br> <span>dict</span>(<span>a</span><span>=</span><span>1</span>, <span>b</span><span>=</span><span>2</span>, <span>c</span><span>=</span><span>3</span>, <span>d</span><span>=</span><span>4</span>),<br> <span>dict</span>(<span>a</span><span>=</span><span>2</span>, <span>b</span><span>=</span><span>3</span>, <span>c</span><span>=</span><span>4</span>, <span>d</span><span>=</span><span>5</span>),<br> <span>dict</span>(<span>a</span><span>=</span><span>3</span>, <span>b</span><span>=</span><span>4</span>, <span>c</span><span>=</span><span>5</span>, <span>d</span><span>=</span><span>6</span>),<br> <span>dict</span>(<span>a</span><span>=</span><span>4</span>, <span>b</span><span>=</span><span>5</span>, <span>c</span><span>=</span><span>6</span>, <span>d</span><span>=</span><span>7</span>)<br>]))<br><span>dtale</span>.<span>show</span>(<span>pd</span>.<span>DataFrame</span>([<span>range</span>(<span>6</span>), <span>range</span>(<span>6</span>), <span>range</span>(<span>6</span>), <span>range</span>(<span>6</span>), <span>range</span>(<span>6</span>), <span>range</span>(<span>6</span>)]), <span>name</span><span>=</span><span>"foo"</span>)</tt></div>
<p dir="auto">This will make the <strong>Instances</strong> button available in all 3 of these D-Tale instances. Clicking that button while in the first instance invoked above will give you this popup:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Instances.png"></a></p>
<p dir="auto">The grid above contains the following information:</p>
<ul dir="auto">
<li>Process: timestamp when the process was started along with the name (if specified in <code>dtale.show()</code>)</li>
<li>Rows: number of rows</li>
<li>Columns: number of columns</li>
<li>Column Names: comma-separated string of column names (only first 30 characters, hover for full listing)</li>
<li>Preview: this button is available any of the non-current instances. Clicking this will bring up left-most 5X5 grid information for that instance</li>
<li>The row highlighted in green signifys the current D-Tale instance</li>
<li>Any other row can be clicked to switch to that D-Tale instance</li>
</ul>
<p dir="auto">Here is an example of clicking the "Preview" button:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Instances_preview.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">About</h4></div>
<p dir="auto">This will give you information about what version of D-Tale you're running as well as if its out of date to whats on PyPi.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Up To Date</th>
<th align="center">Out Of Date</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/About-up-to-date.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/About-out-of-date.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Refresh Widths</h4></div>
<p dir="auto">Mostly a fail-safe in the event that your columns are no longer lining up. Click this and should fix that</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Theme</h4></div>
<p dir="auto">Toggle between light & dark themes for your viewing pleasure (only affects grid, not popups or charts).</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Light</th>
<th align="center">Dark</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/theme/light.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/theme/dark.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Reload Data</h4></div>
<p dir="auto">Force a reload of the data from the server for the current rows being viewing in the grid by clicking this button. This can be helpful when viewing the grid from within another application like jupyter or nested within another website.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Unpin/Pin Menu</h4></div>
<p dir="auto">If you would like to keep your menu pinned to the side of your grid all times rather than always having to click the triaangle in the upper left-hand corner simply click this button. It is persisted back to the server so that it can be applied to all piece of data you've loaded into your session and beyond refreshes.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Language</h4></div>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/chinese_dtale.png"></a></p>
<p dir="auto">I am happy to announce that D-Tale now supports both English & Chinese (there is still more of the translation to be completed but the infrastructure is there). And we are happy to add support for any other languages. Please see instruction on how, here.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Shutdown</h4></div>
<p dir="auto">Pretty self-explanatory, kills your D-Tale session (there is also an auto-kill process that will kill your D-Tale after an hour of inactivity)</p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Column Menu Functions</h3></div>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Col_menu.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Filtering</h4></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=8zo5ZiI1Yzo"></a></p>
<p dir="auto">These interactive filters come in 3 different types: String, Numeric & Date. Note that you will still have the ability to apply custom filters from the "Filter" popup on the main menu, but it will get applied in addition to any column filters.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Type</th>
<th>Filter</th>
<th>Data Types</th>
<th>Features</th>
</tr>
</thead>
<tbody>
<tr>
<td>String</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/filters/string.PNG"></a></td>
<td>strings & booleans</td>
<td>The ability to select multiple values based on what exists in the column. Notice the "Show Missing Only" toggle, this will only show up if your column has nan values</td>
</tr>
<tr>
<td>Date</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/filters/dates.PNG"></a></td>
<td>dates</td>
<td>Specify a range of dates to filter on based on start & end inputs</td>
</tr>
<tr>
<td>Numeric</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/filters/numeric.PNG"></a></td>
<td>ints & floats</td>
<td>For integers the "=" will be similar to strings where you can select multiple values based on what exists in the column. You also have access to other operands: <,>,<=,>=,() - "Range exclusve", [] - "Range inclusive".</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Moving Columns</h4></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=We4TH477rRs"></a></p>
<p dir="auto">All column movements are saved on the server so refreshing your browser won't lose them </p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Hiding Columns</h4></div>
<p dir="auto"><a href="/www.youtube.com/watch?v=ryZT2Lk_YaA"></a></p>
<p dir="auto">All column movements are saved on the server so refreshing your browser won't lose them </p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Delete</h4></div>
<p dir="auto">As simple as it sounds, click this button to delete this column from your dataframe.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Rename</h4></div>
<p dir="auto">Update the name of any column in your dataframe to a name that is not currently in use by your dataframe.</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/rename.png"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Replacements</h4></div>
<p dir="auto">This feature allows users to replace content on their column directly or for safer purposes in a brand new column. Here are the options you have:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Type</th>
<th>Data Types</th>
<th>Description</th>
<th>Menu</th>
</tr>
</thead>
<tbody>
<tr>
<td>Value(s)</td>
<td>all</td>
<td>Replace specific values in a column with raw values, output from another column or an aggregation on your column</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/replacements_value.png"></a></td>
</tr>
<tr>
<td>Spaces Only</td>
<td>strings</td>
<td>Replace string values consisting only of spaces with raw values</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/replacements_spaces.png"></a></td>
</tr>
<tr>
<td>Contains Char/Substring</td>
<td>strings</td>
<td>Replace string values containing a specific character or substring</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/replacements_strings.png"></a></td>
</tr>
<tr>
<td>Scikit-Learn Imputer</td>
<td>numeric</td>
<td>Replace missing values with the output of using different Scikit-Learn imputers like iterative, knn & simple</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/replacements_sklearn.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">Here's a quick demo: <a href="/www.youtube.com/watch?v=GiNFRtcpIt8"></a></p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Lock</h4></div>
<p dir="auto">Adds your column to "locked" columns</p>
<ul dir="auto">
<li>"locked" means that if you scroll horizontally these columns will stay pinned to the right-hand side</li>
<li>this is handy when you want to keep track of which date or security_id you're looking at</li>
<li>by default, any index columns on the data passed to D-Tale will be locked</li>
</ul>
<div dir="auto"><h4 tabindex="-1" dir="auto">Unlock</h4></div>
<p dir="auto">Removed column from "locked" columns</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Sorting</h4></div>
<p dir="auto">Applies/removes sorting (Ascending/Descending/Clear) to the column selected</p>
<p dir="auto"><em>Important</em>: as you add sorts they sort added will be added to the end of the multi-sort. For example:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Action</th>
<th align="center">Sort</th>
</tr>
</thead>
<tbody>
<tr>
<td>click "a"</td>
<td align="center"></td>
</tr>
<tr>
<td>sort asc</td>
<td align="center">a (asc)</td>
</tr>
<tr>
<td>click "b"</td>
<td align="center">a (asc)</td>
</tr>
<tr>
<td>sort desc</td>
<td align="center">a (asc), b(desc)</td>
</tr>
<tr>
<td>click "a"</td>
<td align="center">a (asc), b(desc)</td>
</tr>
<tr>
<td>sort None</td>
<td align="center">b(desc)</td>
</tr>
<tr>
<td>sort desc</td>
<td align="center">b(desc), a(desc)</td>
</tr>
<tr>
<td>click "X" on sort display</td>
<td align="center"></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h4 tabindex="-1" dir="auto">Formats</h4></div>
<p dir="auto">Apply simple formats to numeric values in your grid</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Type</th>
<th align="center">Editing</th>
<th align="center">Result</th>
</tr>
</thead>
<tbody>
<tr>
<td>Numeric</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Formatting_apply.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Post_formatting.png"></a></td>
</tr>
<tr>
<td>Date</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Formatting_date_apply.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Post_date_formatting.png"></a></td>
</tr>
<tr>
<td>String</td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Formatting_string_apply.png"></a></td>
<td align="center"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/Post_string_formatting.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">For all data types you have the ability to change what string is ued for display.</p>
<p dir="auto">For numbers here's a grid of all the formats available with -123456.789 as input:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Format</th>
<th align="center">Output</th>
</tr>
</thead>
<tbody>
<tr>
<td>Precision (6)</td>
<td align="center">-123456.789000</td>
</tr>
<tr>
<td>Thousands Sep</td>
<td align="center">-123,456.789</td>
</tr>
<tr>
<td>Abbreviate</td>
<td align="center">-123k</td>
</tr>
<tr>
<td>Exponent</td>
<td align="center">-1e+5</td>
</tr>
<tr>
<td>BPS</td>
<td align="center">-1234567890BPS</td>
</tr>
<tr>
<td>Red Negatives</td>
<td align="center"><span>-123457</span></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto">For strings you can apply the follwoing formats:</p>
<ul dir="auto">
<li><strong>Truncation:</strong> truncate long strings to a certain number of characters and replace with an allipses "..." and see the whole value on hover.</li>
<li><strong>Hyperlinks:</strong> If your column is comprised of URL strings you can make them hyperlinks which will open a new tab</li>
</ul>
<div dir="auto"><h4 tabindex="-1" dir="auto">Describe (Column Analysis)</h4></div>
<p dir="auto">Based on the data type of a column different charts will be shown. This side panel can be closed using the 'X' button in the upper right or by pressing the ESC key.</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Chart</th>
<th>Data Types</th>
<th>Sample</th>
</tr>
</thead>
<tbody>
<tr>
<td>Box Plot</td>
<td>Float, Int, Date</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/boxplot.png"></a></td>
</tr>
<tr>
<td>Histogram</td>
<td>Float, Int</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/histogram.PNG"></a></td>
</tr>
<tr>
<td>Value Counts</td>
<td>Int, String, Bool, Date, Category</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/value_counts.PNG"></a></td>
</tr>
<tr>
<td>Word Value Counts</td>
<td>String</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/word_value_counts.png"></a></td>
</tr>
<tr>
<td>Category</td>
<td>Float</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/category.PNG"></a></td>
</tr>
<tr>
<td>Geolocation*</td>
<td>Int, Float</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/geolocation.PNG"></a></td>
</tr>
<tr>
<td>Q-Q Plot</td>
<td>Int, Float, Date</td>
<td><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/qq.png"></a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto"><strong>Histogram</strong> can be displayed in any number of bins (default: 20), simply type a new integer value in the bins input</p>
<p dir="auto"><strong>Value Count</strong> by default, show the top 100 values ranked by frequency. If you would like to show the least frequent values simply make your number negative (-10 => 10 least frequent value)</p>
<p dir="auto"><strong>Value Count w/ Ordinal</strong> you can also apply an ordinal to your <strong>Value Count</strong> chart by selecting a column (of type int or float) and applying an aggregation (default: sum) to it (sum, mean, etc...) this column will be grouped by the column you're analyzing and the value produced by the aggregation will be used to sort your bars and also displayed in a line. Here's an example:</p>
<p dir="auto"><a href="/raw.githubusercontent.com/aschonfeld/dtale-media/master/images/analysis/value_counts_ordinal.PNG"></a></p>
<p dir="auto"><strong>Word Value Count</strong> you can analyze string data by splitting each record by spaces to see the counts of each word. This chart has all the same functions available as "Value Counts". In addition, you can select multiple "Cleaner" functions to be applied to your column before building word values. These functions will perform operations like removing punctuation, removing numeric character & normalizing accent characters.</p>
<p dir="auto"><strong>Category (Category Breakdown)</strong> when viewing float columns you can also see them broken down by a categorical column (string, date, int, etc...). This means that when you select a category column this will then display the frequency of each category in a line as well as bars based on the float column you're analyzing grouped by that category and computed by your aggregation (default: mean).</p>
<p dir="auto"><strong>Geolocation</strong> when your data contains latitude & longitude then you can view the coverage in a plotly scattergeo map. In order to have access this chart your dataframe must have at least one of each of these types of columns:</p>
<ul dir="auto">
<li>"lat" must be contained within the lower-cased version of the column name and values be between -90 & 90</li>
<li>"lon" must be contained within the lower-cased version of the column name and values be between -180 & 180</li>
</ul>
<div dir="auto"><h3 tabindex="-1" dir="auto">Hotkeys</h3></div>
<p dir="auto">These are key combinations you can use in place of clicking actual buttons to save a little time:</p>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th>Keymap</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>shift+m</code></td>
<td>Opens main menu*</td>
</tr>
<tr>
<td><code>shift+d</code></td>
<td>Opens "Describe" page*</td>
</tr>
<tr>
<td><code>shift+f</code></td>
<td>Opens "Custom Filter"*</td>
</tr>
<tr>
<td><code>shift+b</code></td>
<td>Opens "Build Column"*</td>
</tr>
<tr>
<td><code>shift+c</code></td>
<td>Opens "Charts" page*</td>
</tr>
<tr>
<td><code>shift+x</code></td>
<td>Opens "Code Export"*</td>
</tr>
<tr>
<td><code>esc</code></td>
<td>Closes any open modal window or side panel & exits cell editing</td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<p dir="auto"><code>*</code> Does not fire if user is actively editing a cell.</p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Menu Functions Depending on Browser Dimensions</h3></div>
<p dir="auto">Depending on the dimensions of your browser window the following buttons will not open modals, but rather separate browser windows: Correlations, Describe & Instances (see images from Jupyter Notebook, also Charts will always open in a separate browser window)</p>
<div dir="auto"><h2 tabindex="-1" dir="auto">For Developers</h2></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Cloning</h3></div>
<p dir="auto">Clone the code (git clone ssh://<a href="/mailto:git@github.com">git@github.com</a>:manahl/dtale.git), then start the backend server:</p>
<div dir="auto"><tt>$ git clone ssh://git@github.com:manahl/dtale.git<br><span><span>#</span> install the dependencies</span><br>$ python setup.py develop<br><span><span>#</span> start the server</span><br>$ dtale --csv-path /home/jdoe/my_csv.csv --csv-parse_dates date</tt></div>
<p dir="auto">You can also run dtale from PyDev directly.</p>
<p dir="auto">You will also want to import javascript dependencies and build the source (all javascript code resides in the <code>frontend</code> folder):</p>
<div dir="auto"><tt>$ <span>cd</span> frontend<br>$ npm install<br><span><span>#</span> 1) a persistent server that serves the latest JS:</span><br>$ npm run watch<br><span><span>#</span> 2) or one-off build:</span><br>$ npm run build</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Running Tests</h3></div>
<p dir="auto">D-Tale has both a React front-end and a Flask/Python back-end, each with their own test suites.</p>
<div dir="auto"><h4 tabindex="-1" dir="auto">Front-End Tests (JavaScript/React)</h4></div>
<p dir="auto">Front-end tests are located in <code>frontend/static/__tests__/</code> and use Jest.</p>
<p dir="auto"><strong>Node.js version:</strong> CI uses Node.js <strong>24.4.0</strong>. To check your version:</p>
<div dir="auto"><tt>$ node --version<br>$ npm --version<br>$ yarn --version</tt></div>
<p dir="auto"><strong>Install dependencies:</strong></p>
<div dir="auto"><tt>$ <span>cd</span> frontend<br>$ yarn install --frozen-lockfile</tt></div>
<p dir="auto"><strong>Run all front-end tests:</strong></p>
<div dir="auto"><tt>$ <span>cd</span> frontend<br>$ npm <span>test</span></tt></div>
<p dir="auto"><strong>Run tests with coverage:</strong></p>
<div dir="auto"><tt>$ <span>cd</span> frontend<br>$ yarn run test-with-coverage -w 1</tt></div>
<p dir="auto"><strong>Run individual test files:</strong></p>
<div dir="auto"><tt>$ <span>cd</span> frontend<br>$ npm run <span>test</span> -- static/__tests__/dtale/DataViewer-base-test.jsx</tt></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">Back-End Tests (Python)</h4></div>
<p dir="auto">Back-end tests are located in the <code>tests/</code> directory and use pytest.</p>
<p dir="auto"><strong>Set up a virtual environment and install dependencies:</strong></p>
<div dir="auto"><tt>$ virtualenv venv<br>$ <span>source</span> venv/bin/activate <span><span>#</span> On Windows: venv\Scripts\activate</span><br>$ pip install -r requirements.txt<br>$ python setup.py develop</tt></div>
<p dir="auto"><strong>Install test dependencies:</strong></p>
<div dir="auto"><tt><span><span>#</span> Core test dependencies</span><br>$ pip install -e <span><span>"</span>.[tests]<span>"</span></span><br><br><span><span>#</span> Optional feature test dependencies</span><br>$ pip install -e <span><span>"</span>.[arcticdb]<span>"</span></span> <span><span>#</span> ArcticDB integration tests</span><br>$ pip install -e <span><span>"</span>.[dash-bio]<span>"</span></span> <span><span>#</span> Dash Bio component tests</span><br>$ pip install -e <span><span>"</span>.[ngrok]<span>"</span></span> <span><span>#</span> ngrok tunnel tests</span><br>$ pip install -e <span><span>"</span>.[redis]<span>"</span></span> <span><span>#</span> Redis storage backend tests</span><br>$ pip install -e <span><span>"</span>.[r]<span>"</span></span> <span><span>#</span> R integration tests (requires R to be installed)</span></tt></div>
<p dir="auto"><strong>Installing R (required for R integration tests):</strong></p>
<div dir="auto"><tt><span><span>#</span> Ubuntu/Debian</span><br>$ sudo apt update<br>$ sudo apt -y install r-base<br><br><span><span>#</span> macOS (using Homebrew)</span><br>$ brew install r</tt></div>
<p dir="auto"><strong>Run all back-end tests:</strong></p>
<div dir="auto"><tt>$ pytest tests -v</tt></div>
<p dir="auto"><strong>Run tests with coverage:</strong></p>
<div dir="auto"><tt>$ pytest tests --cov dtale --cov-report html --cov-report xml -v</tt></div>
<p dir="auto"><strong>Run a specific test file:</strong></p>
<div dir="auto"><tt>$ pytest tests/dtale/test_views.py -v</tt></div>
<p dir="auto"><strong>Run a specific test function:</strong></p>
<div dir="auto"><tt>$ pytest tests/dtale/test_views.py::test_head_data_id -v</tt></div>
<div dir="auto"><h4 tabindex="-1" dir="auto">Running All Tests (CI-style)</h4></div>
<p dir="auto">To run both front-end and back-end tests as they run in CI:</p>
<div dir="auto"><tt><span><span>#</span> Build front-end first (required for back-end tests)</span><br>$ <span>cd</span> frontend<br>$ yarn install --frozen-lockfile<br>$ yarn run build<br>$ <span>cd</span> ..<br><br><span><span>#</span> Install all optional dependencies</span><br>$ <span>source</span> venv/bin/activate<br>$ pip install -e <span><span>"</span>.[arcticdb,dash-bio,ngrok,redis,r,tests]<span>"</span></span><br><br><span><span>#</span> Run back-end tests</span><br>$ pytest tests --cov dtale -v</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Linting</h3></div>
<p dir="auto">You can lint all the JS and CSS to confirm there's nothing obviously wrong with
it:</p>
<div dir="auto"><tt>$ npm run lint</tt></div>
<p dir="auto">You can also lint individual JS files:</p>
<div dir="auto"><tt>$ npm run lint-js-file -s -- static/dtale/DataViewer.jsx</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Formatting JS</h3></div>
<p dir="auto">You can auto-format code as follows:</p>
<div dir="auto"><tt>$ npm run format</tt></div>
<div dir="auto"><h3 tabindex="-1" dir="auto">Docker Development</h3></div>
<p dir="auto">You can build python 27-3 & run D-Tale as follows:</p>
<div dir="auto"><tt>$ yarn run build<br>$ docker-compose build dtale_2_7<br>$ docker run -it --network host dtale_2_7:latest<br>$ python<br>>>> import pandas as pd<br>>>> df = pd.DataFrame([dict(a=1,b=2,c=3)])<br>>>> import dtale<br>>>> dtale.show(df)</tt></div>
<p dir="auto">Then view your D-Tale instance in your browser using the link that gets printed</p>
<p dir="auto">You can build python 36-1 & run D-Tale as follows:</p>
<div dir="auto"><tt>$ yarn run build<br>$ docker-compose build dtale_3_6<br>$ docker run -it --network host dtale_3_6:latest<br>$ python<br>>>> import pandas as pd<br>>>> df = pd.DataFrame([dict(a=1,b=2,c=3)])<br>>>> import dtale<br>>>> dtale.show(df)</tt></div>
<p dir="auto">Then view your D-Tale instance in your browser using the link that gets printed</p>
<div dir="auto"><h3 tabindex="-1" dir="auto">Adding Language Support</h3></div>
<p dir="auto">Currently D-Tale support both english & chinese but other languages will gladly be supported. To add another language simply open a pull request with the following:</p>
<ul dir="auto">
<li>cake a copy & translate the values in the following JSON english JSON files and save them to the same locations as each file</li>
<li><a href="/github.com/man-group/dtale/blob/master/dtale/translations/en.json">Back-End</a></li>
<li><a href="/github.com/man-group/dtale/blob/master/static/translations/en.json">Front-End</a></li>
<li>please make the name of these files the name of the language you are adding (currently english -> en, chinese -> cn)</li>
<li>be sure to keep the keys in english, that is important</li>
</ul>
<p dir="auto">Looking forward to what languages come next! </p>
<div dir="auto"><h2 tabindex="-1" dir="auto">Global State/Data Storage</h2></div>
<p dir="auto">If D-Tale is running in an environment with multiple python processes (ex: on a web server running <a href="/github.com/benoitc/gunicorn">gunicorn</a>) it will most likely encounter issues with inconsistent state. Developers can fix this by configuring the system D-Tale uses for storing data. Detailed documentation is available here: <a href="/github.com/man-group/dtale/blob/master/docs/GLOBAL_STATE.md">Data Storage and managing Global State</a></p>
<div dir="auto"><h2 tabindex="-1" dir="auto">Startup Behavior</h2></div>
<p dir="auto">Here's a little background on how the <code>dtale.show()</code> function works:</p>
<ul dir="auto">
<li>by default it will look for ports between 40000 & 49000, but you can change that range by specifying the environment variables DTALE_MIN_PORT & DTALE_MAX_PORT</li>
<li>think of sessions as python consoles or jupyter notebooks</li>
</ul>
<ol dir="auto">
<li>Session 1 executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="2" dir="auto">
<li>Session 1 executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40000</td>
<td align="center">1,2</td>
<td align="center"><a href="/localhost:40000/dtale/main/%5B1,2%5D">http://localhost:40000/dtale/main/[1,2]</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="2" dir="auto">
<li>Session 2 executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40000</td>
<td align="center">1,2</td>
<td align="center"><a href="/localhost:40000/dtale/main/%5B1,2%5D">http://localhost:40000/dtale/main/[1,2]</a></td>
</tr>
<tr>
<td align="center">2</td>
<td align="center">40001</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40001/dtale/main/1">http://localhost:40001/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="3" dir="auto">
<li>Session 1 executes <code>dtale.show(df, port=40001, force=True)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">1,2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B1,2,3%5D">http://localhost:40001/dtale/main/[1,2,3]</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="4" dir="auto">
<li>Session 3 executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">1,2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B1,2,3%5D">http://localhost:40001/dtale/main/[1,2,3]</a></td>
</tr>
<tr>
<td align="center">3</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="5" dir="auto">
<li>Session 2 executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">1,2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B1,2,3%5D">http://localhost:40001/dtale/main/[1,2,3]</a></td>
</tr>
<tr>
<td align="center">3</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">2</td>
<td align="center">40002</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40002/dtale/main/1">http://localhost:40002/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="6" dir="auto">
<li>Session 4 executes <code>dtale.show(df, port=8080)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">1,2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B1,2,3%5D">http://localhost:40001/dtale/main/[1,2,3]</a></td>
</tr>
<tr>
<td align="center">3</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">2</td>
<td align="center">40002</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40002/dtale/main/1">http://localhost:40002/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">4</td>
<td align="center">8080</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:8080/dtale/main/1">http://localhost:8080/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="7" dir="auto">
<li>Session 1 executes <code>dtale.get_instance(1).kill()</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data IDs</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B2,3%5D">http://localhost:40001/dtale/main/[2,3]</a></td>
</tr>
<tr>
<td align="center">3</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">2</td>
<td align="center">40002</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40002/dtale/main/1">http://localhost:40002/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">4</td>
<td align="center">8080</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:8080/dtale/main/1">http://localhost:8080/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<ol start="7" dir="auto">
<li>Session 5 sets DTALE_MIN_RANGE to 30000 and DTALE_MAX_RANGE 39000 and executes <code>dtale.show(df)</code> our state is:</li>
</ol>
<markdown-accessiblity-table><table>
<thead>
<tr>
<th align="center">Session</th>
<th align="center">Port</th>
<th align="center">Active Data ID(s)</th>
<th align="center">URL(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">1</td>
<td align="center">40001</td>
<td align="center">2,3</td>
<td align="center"><a href="/localhost:40001/dtale/main/%5B2,3%5D">http://localhost:40001/dtale/main/[2,3]</a></td>
</tr>
<tr>
<td align="center">3</td>
<td align="center">40000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40000/dtale/main/1">http://localhost:40000/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">2</td>
<td align="center">40002</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:40002/dtale/main/1">http://localhost:40002/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">4</td>
<td align="center">8080</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:8080/dtale/main/1">http://localhost:8080/dtale/main/1</a></td>
</tr>
<tr>
<td align="center">5</td>
<td align="center">30000</td>
<td align="center">1</td>
<td align="center"><a href="/localhost:30000/dtale/main/1">http://localhost:30000/dtale/main/1</a></td>
</tr>
</tbody>
</table></markdown-accessiblity-table>
<div dir="auto"><h2 tabindex="-1" dir="auto">Documentation</h2></div>
<p dir="auto">Have a look at the <a href="/dtale.readthedocs.io">detailed documentation</a>.</p>
<div dir="auto"><h2 tabindex="-1" dir="auto">Dependencies</h2></div>
<ul dir="auto">
<li>Back-end
<ul dir="auto">
<li>dash</li>
<li>dash_daq</li>
<li>Flask</li>
<li>Flask-Compress</li>
<li>Pandas</li>
<li>plotly</li>
<li>scikit-learn</li>
<li>scipy</li>
<li>xarray</li>
<li>arctic [extra]</li>
<li>dash-bio [extra]</li>
<li>flask-ngrok [extra]</li>
<li>redis [extra]</li>
<li>rpy2 [extra]</li>
<li>streamlit [extra]</li>
<li>swifter [extra]</li>
</ul>
</li>
<li>Front-end
<ul dir="auto">
<li>react-virtualized</li>
<li>chart.js</li>
</ul>
</li>
</ul>
<div dir="auto"><h2 tabindex="-1" dir="auto">Acknowledgements</h2></div>
<p dir="auto">D-Tale has been under active development at <a href="/www.numeric.com/">Man Numeric</a> since 2019.</p>
<p dir="auto">Original concept and implementation: <a href="/github.com/aschonfeld">Andrew Schonfeld</a></p>
<p dir="auto">Contributors:</p>
<ul dir="auto">
<li><a href="/github.com/phillipdupuis">Phillip Dupuis</a></li>
<li><a href="/github.com/fersarr">Fernando Saravia Rajal</a></li>
<li><a href="/github.com/DominikMChrist">Dominik Christ</a></li>
<li><a href="/github.com/reza1615">Reza Moshksar</a></li>
<li><a href="/github.com/bnouvelbmll">Bertrand Nouvel</a></li>
<li><a href="/github.com/cboddy">Chris Boddy</a></li>
<li><a href="/github.com/jasonkholden">Jason Holden</a></li>
<li><a href="/github.com/TomTaylorLondon">Tom Taylor</a></li>
<li><a href="/github.com/Wilfred">Wilfred Hughes</a></li>
<li>Mike Kelly</li>
<li><a href="/github.com/vincentriemer">Vincent Riemer</a></li>
<li><a href="/youssef-habchi.com/">Youssef Habchi</a> - title font</li>
<li><a href="/github.com/yadhukrishnam">Yadhu Krishna</a></li>
<li>... and many others ...</li>
</ul>
<p dir="auto">Contributions welcome!</p>
<div dir="auto"><h2 tabindex="-1" dir="auto">License</h2></div>
<p dir="auto">D-Tale is licensed under the GNU LGPL v2.1. A copy of which is included in <a href="/github.com/man-group/dtale/blob/master/LICENSE">LICENSE</a></p>
</article></div></div></div></div></div> <!-- --> <!-- --> </div>
</react-partial>
<input type="hidden" value="yvGpAR/dOQKGc0hHyHmOq4A6KMy1076fNOUa0cXYcNplkbeTbb/zu Csk2VkaSkaJOHz+Rgn9D+F5eWo87hAXHg==" />
</div>
<div>
<div>
<div>
<div>
<div>
<h2>About</h2>
<p>
Visualizer for pandas data structures
</p>
<div>
<span>
<a href="/alphatechadmin.pythonanywhere.com">alphatechadmin.pythonanywhere.com</a>
</span>
</div>
<h3>Topics</h3>
<div>
<div>
<a href="/github.com/topics/react">
react
</a>
<a href="/github.com/topics/visualization">
visualization
</a>
<a href="/github.com/topics/flask">
flask
</a>
<a href="/github.com/topics/data-science">
data-science
</a>
<a href="/github.com/topics/ipython">
ipython
</a>
<a href="/github.com/topics/jupyter-notebook">
jupyter-notebook
</a>
<a href="/github.com/topics/pandas">
pandas
</a>
<a href="/github.com/topics/data-visualization">
data-visualization
</a>
<a href="/github.com/topics/python3">
python3
</a>
<a href="/github.com/topics/xarray">
xarray
</a>
<a href="/github.com/topics/data-analysis">
data-analysis
</a>
<a href="/github.com/topics/python27">
python27
</a>
<a href="/github.com/topics/react-virtualized">
react-virtualized
</a>
<a href="/github.com/topics/plotly-dash">
plotly-dash
</a>
</div>
</div>
<h3>Resources</h3>
<div>
Readme
</div>
<h3>License</h3>
<div>
LGPL-2.1 license
</div>
<h3>Security policy</h3>
<div>
Security policy
</div>
<include-fragment src="/man-group/dtale/hovercards/citation/sidebar_partial?tree_name=master">
<div data-show-on-forbidden-error hidden>
<div>
<div>
<div>
<h3> Uh oh!
</h3>
<p> <p>There was an error while loading. <a href="" aria-label="Please reload this page">Please reload this page</a>.</p>
</p>
</div> </div>
</div> </div>
</include-fragment>
<div>
<a href="/github.com/man-group/dtale/activity">
<span>Activity</span></a> </div>
<div>
<a href="/github.com/man-group/dtale/custom-properties">
<span>Custom properties</span></a> </div>
<h3>Stars</h3>
<div>
<a href="/github.com/man-group/dtale/stargazers">
<strong>5.1k</strong>
stars</a> </div>
<h3>Watchers</h3>
<div>
<a href="/github.com/man-group/dtale/watchers">
<strong>67</strong>
watching</a> </div>
<h3>Forks</h3>
<div>
<a href="/github.com/man-group/dtale/forks">
<strong>431</strong>
forks</a> </div>
<div>
<a href="/github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fman-group%2Fdtale&report=man-group+%28user%29">
Report repository
</a> </div>
</div>
</div>
</div>
<div>
<div>
<h2>
<a href="/github.com/man-group/dtale/releases">Releases
<span title="182">182</span></a></h2>
<a href="/github.com/man-group/dtale/releases/tag/v3.19.1">
<div>
<div>
<span>D-Tale 3.19.1</span>
<span title="Label: Latest">
Latest
</span> </div>
<div><relative-time datetime="2026-01-28T19:17:40Z">Jan 28, 2026</relative-time></div>
</div>
</a> <div>
<a href="/github.com/man-group/dtale/releases">+ 181 releases</a></div>
</div>
</div>
<div>
<div>
<h2>
<a href="/github.com/orgs/man-group/packages?repo_name=dtale">Packages
<span title="0" hidden="hidden">0</span></a></h2>
<div >
No packages published <br>
</div>
</div>
</div>
<div hidden>
<div>
<include-fragment src="/man-group/dtale/used_by_list" accept="text/fragment+html">
<div data-show-on-forbidden-error hidden>
<div>
<div>
<div>
<h3> Uh oh!
</h3>
<p> <p>There was an error while loading. <a href="" aria-label="Please reload this page">Please reload this page</a>.</p>
</p>
</div> </div>
</div> </div>
</include-fragment>
</div>
</div>
<div>
<div>
<h2>
<a href="/github.com/man-group/dtale/graphs/contributors">Contributors
<span title="26">26</span></a></h2>
<ul>
<li
>
<a href="/github.com/aschonfeld">
</a>
</li>
<li
>
<a href="/github.com/AnthraX1">
</a>
</li>
<li
>
<a href="/github.com/apps/dependabot">
</a>
</li>
<li
>
<a href="/github.com/matrach">
</a>
</li>
<li
>
<a href="/github.com/jasonkholden">
</a>
</li>
<li
>
<a href="/github.com/mswastik">
</a>
</li>
<li
>
<a href="/github.com/appznoix">
</a>
</li>
<li
>
<a href="/github.com/poodlewars">
</a>
</li>
<li
>
<a href="/github.com/aflag">
</a>
</li>
<li
>
<a href="/github.com/dsblank">
</a>
</li>
<li
>
<a href="/github.com/msabramo">
</a>
</li>
<li
>
<a href="/github.com/astrojuanlu">
</a>
</li>
<li
>
<a href="/github.com/cdeil">
</a>
</li>
<li
>
<a href="/github.com/thekchang">
</a>
</li>
</ul>
<div>
<a href="/github.com/man-group/dtale/graphs/contributors">+ 12 contributors</a></div>
</div>
</div>
<div>
<div>
<h2>Languages</h2>
<div>
<span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
<span itemprop="keywords"></span>
</span></div>
<ul>
<li>
<a href="/github.com/man-group/dtale/search?l=typescript">
<span>TypeScript</span>
<span>48.2%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=python">
<span>Python</span>
<span>43.1%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=css">
<span>CSS</span>
<span>6.6%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=javascript">
<span>JavaScript</span>
<span>1.1%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=html">
<span>HTML</span>
<span>0.6%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=dockerfile">
<span>Dockerfile</span>
<span>0.2%</span>
</a>
</li>
<li>
<a href="/github.com/man-group/dtale/search?l=scss">
<span>SCSS</span>
<span>0.2%</span>
</a>
</li>
</ul>
</div>
</div>
</div>
</div>
</div></div>
</div>
</div>
</turbo-frame>
</main>
</div>
</div>
<footer role="contentinfo" >
<h2>Footer</h2>
<div>
<div>
<a href="/github.com">
</a>
<span>
(c) 2026 GitHub, Inc.
</span>
</div>
<nav aria-label="Footer">
<h3 id="sr-footer-heading">Footer navigation</h3>
<ul aria-labelledby="sr-footer-heading">
<li>
<a href="/docs.github.com/site-policy/github-terms/github-terms-of-service">Terms</a>
</li>
<li>
<a href="/docs.github.com/site-policy/privacy-policies/github-privacy-statement">Privacy</a>
</li>
<li>
<a href="/github.com/security">Security</a>
</li>
<li>
<a href="/www.githubstatus.com/">Status</a>
</li>
<li>
<a href="/github.community/">Community</a>
</li>
<li>
<a href="/docs.github.com/">Docs</a>
</li>
<li>
<a href="/support.github.com?tags=dotcom-footer">Contact</a>
</li>
<li >
<cookie-consent-link>
<button
type="button"
>
Manage cookies
</button>
</cookie-consent-link>
</li>
<li>
<cookie-consent-link>
<button
type="button"
>
Do not share my personal information
</button>
</cookie-consent-link>
</li>
</ul>
</nav>
</div>
</footer>
<ghcc-consent id="ghcc"
></ghcc-consent>
<div id="ajax-error-message" hidden>
<button type="button" aria-label="Dismiss error">
<svg aria-hidden="true" height="16" viewBox="0 0 16 16" version="1.1" width="16">
<path d="M3.72 3.72a.75.75 0 0 1 1.06 0L8 6.94l3.22-3.22a.749.749 0 0 1 1.275.326.749.749 0 0 1-.215.734L9.06 8l3.22 3.22a.749.749 0 0 1-.326 1.275.749.749 0 0 1-.734-.215L8 9.06l-3.22 3.2
</body>
</html> |
