<jupyter_start><jupyter_text>`bulkdata` Usage The `bulkdata` package loads BDF files into memory as `Deck` objects,which are collections of `Card` objects. This notebook willdemonstrate how to create, modify, and utilize these objects.  Card Build cardFirst, let's initialize a `Card` object with name "EXAMPLE".<jupyter_code>from bulkdata import Card

card = Card("EXAMPLE")<jupyter_output><empty_output><jupyter_text>At this point, the card only has a name and no fields. If we dump the card to its string representation in _fixed_ format with `dumps`, we get:<jupyter_code>print(card.dumps("fixed"))<jupyter_output>EXAMPLE<jupyter_text>That's not very useful, let's enter some fields into the card.<jupyter_code># append integer field
card.append(100)

# append real field
card.append(3.14)

# append character field
card.append("string")<jupyter_output><empty_output><jupyter_text>Now if we look at the card's string representation we see:<jupyter_code>print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string<jupyter_text>We can also enter blank fields.<jupyter_code># append a blank field using None
card.append(None)

# append a blank field using null string
card.append("")

# trailing blank fields are ingored during `dumps` call, 
# so printing the card here yields the same result as the
# previous print
print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string<jupyter_text>Using the `append` method, we entered one field at a time. But what if we want to enter a list of fields? This is done with the `extend` method:<jupyter_code># append a list of integer fields
card.extend([0, 1, 2, 3, 4])

# NOTE: blank fields are there
print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4<jupyter_text>Sometimes, field entries span across two fields to allow more characters (this is particularly common in __ZAERO__, where the _Large Field_ format doesn't exist). Since it's technically a single entry, we use the `append` method to do this while specifying the need for 2 fields, instead of the default 1.<jupyter_code># append a character field spanning 2 field cells
card.append("thisislongstring", fieldspan=2)

print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4       thisislongstring<jupyter_text>There are also times when two lists of field entries have alternating positions in the card. In this case, the easiest way to enter the fields is with a little help from the builtin `zip` function.<jupyter_code># append two field lists, alternating
numbers = [42, -9.99999e9, 10000000, -.0000000001]
strings = ["one", "two", "three", "four"]
for number, string in zip(numbers, strings):
    card.append(number)
    card.append(string)

print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4       thisislongstring42      one     -10.+9  two     +1      
+1      10000000three   -1.-10  four<jupyter_text>And if each field entry spans across two fields:<jupyter_code>for longstring in ["123456789", "helloworld"]:
    card.append(longstring, fieldspan=2)

print(card.dumps("fixed"))<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4       thisislongstring42      one     -10.+9  two     +1      
+1      10000000three   -1.-10  four    123456789       helloworld<jupyter_text>By the way, we can also get the card's _free_ format representation:<jupyter_code>print(card.dumps("free"))<jupyter_output>EXAMPLE,100,3.14,string, , ,0,1,2,+0
+0,3,4,thisislo,ngstring,42,one,-10.+9,two,+1
+1,10000000,three,-1.-10,four,12345678,9,hellowor,ld<jupyter_text>Printing the card object uses the `dumps` method, which defaults to _fixed_ format if no format argument is provided.<jupyter_code># these are all analogous

# print(card.dumps("fixed"))
# print(card.dumps())
print(card)<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4       thisislongstring42      one     -10.+9  two     +1      
+1      10000000three   -1.-10  four    123456789       helloworld<jupyter_text>Modify card One of the main benefits of `bulkdata` is the ability to edit existing cards, agnostic to card field specifications and/or how the card was built.Let's make some edits to the card we created to demonstrate what we mean. First, a simple edit to the first field of the card.<jupyter_code># set first field value to 99
print("First field current:", card[0])

card[0] = 99

print("First field set to 99:", card[0])

# increment first field value by 100
card[0] += 100

print("First field increment by 100:", card[0])<jupyter_output>First field current: 100
First field set to 99: 99
First field increment by 100: 199<jupyter_text>Now let's update the two blank fields we set earlier to contain a character entry spanning two fields.<jupyter_code># the blank fields are at index 3 and 4
print("Blank fields:", card[[3, 4]])

card[[3, 4]] = "newstr"

print("One field no longer blank:", card[[3, 4]])

card[[3, 4]] = "newlongstring"

print("Both fields no longer blank:", card[[3, 4]])<jupyter_output>Blank fields: ['', '']
One field no longer blank: ['newstr', '']
Both fields no longer blank: ['newlongs', 'tring']<jupyter_text>Note that when we specify several indexes during the set operation, every field at that index will be cleared to make way for the new value; if the new field value does not cover every field, the leftover fields will remain blank after the set. In the first set above, the "newstr" did not required two fields, but because we specified that both index 3 and 4 fields were being set, the second field (at index 4) remained blank after.This should make clear that the way fields are entered does not matter, internally the card maintains a value for each field cell. The `Card` object handles the conversion of field inputs to the appropriate field cells according to the specified operation.Let's do a similar operation but with a list of new field values and indexing the set operation with slice syntax.<jupyter_code># we will overwrite the fields from index 10 to 16 (excluding 16)
print("Fields to overwrite:", card[10:16])

card[10:16] = [5, 6, 7, 8, 9]

print("Fields after set:", card[10:16])<jupyter_output>Fields to overwrite: ['thisislo', 'ngstring', 42, 'one', -10000000000.0, 'two']
Fields after set: [5, 6, 7, 8, 9, '']<jupyter_text>To remove fields, we can use the `pop` method the remove the last field...<jupyter_code>print("Last line before pop:", card[16:])

popped_field = card.pop()

print("Popped field:", popped_field)
print("Last line after pop:", card[16:])<jupyter_output>Last line before pop: [10000000, 'three', -1e-10, 'four', 12345678, 9, 'hellowor', 'ld']
Popped field: ld
Last line after pop: [10000000, 'three', -1e-10, 'four', 12345678, 9, 'hellowor']<jupyter_text>... or `del` to remove at specified index(s)...<jupyter_code># delete first item of last (3rd) line
del card[16]

print("Last line after delete first:", card[16:])

# delete remaining first two items of last line
del card[16:18]

print("Last line after delete remaining first & second:", card[16:])<jupyter_output>Last line after delete first: ['three', -1e-10, 'four', 12345678, 9, 'hellowor']
Last line after delete remaining first & second: ['four', 12345678, 9, 'hellowor']<jupyter_text>... or `resize`, which removes fields (or appends blank fields) until the specified size is reached.<jupyter_code>card.resize(16)

print("Last line after resize:", card[16:])
print("Number of fields:", len(card))<jupyter_output>Last line after resize: []
Number of fields: 16<jupyter_text>After these modifications, we can see that the card has been updated:<jupyter_code>print(card)<jupyter_output>EXAMPLE 199     3.14    string  newlongstring   0       1       2       +0      
+0      3       4       5       6       7       8       9<jupyter_text>The card is functionally analogous to a `list` of field values.<jupyter_code>print("Number of fields:", len(card))
print("Card field values:", card[:])<jupyter_output>Number of fields: 16
Card field values: [199, 3.14, 'string', 'newlongs', 'tring', 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, '']<jupyter_text>Alternate way of building a cardIf we have an idea of how many fields we need, we can alternatively initialize the card with a specified number of blank fields and then overwrite the fields with set operations, instead of using `append` and/or `extend`.<jupyter_code># if we don't know the exact number of fields, 
# we can overestimate for now and remove the excess later
numfields = 100
card = Card("EXAMPLE", size=numfields)

# indeed, our card has 100 fields
print(len(card))<jupyter_output>100<jupyter_text>Let's make this card just like the one from the __Build card__ section<jupyter_code># set integer field
card[0] = 100

# set real field
card[1] = 3.14

# set character field
card[2] = "string"

# # set blank field using None
# card[3] = None

# # set blank field using null string
# card[4] = ""

# ^ that would be redundant, fields are already blank

# set list of integer fields
card[5:10] = [0, 1, 2, 3, 4]

# set character field spanning 2 field cells
card[10:12] = "thisislongstring"

# set two field lists, alternating
numbers = [42, -9.99999e9, 10000000, -.0000000001]
strings = ["one", "two", "three", "four"]
card[12:20:2] = numbers
card[13:20:2] = strings

# set field list with fields spanning 2 field cells
for i, longstring in enumerate(["123456789", "helloworld"]):
    i0 = 20 + 2*i
    i1 = i0 + 2
    card[i0:i1] = longstring

# remove the trailing blank fields
print("Number of fields before strip:", len(card))
card.strip()
print("Number of fields after strip:", len(card))<jupyter_output>Number of fields before strip: 100
Number of fields after strip: 24<jupyter_text>The string representation of the card should look familar.<jupyter_code>print(card)<jupyter_output>EXAMPLE 100     3.14    string                  0       1       2       +0      
+0      3       4       thisislongstring42      one     -10.+9  two     +1      
+1      10000000three   -1.-10  four    123456789       helloworld<jupyter_text>For more information on the `Card` class, check out the API documentation.  DeckThe main utility of `bulkdata` is the ability to load an entire BDF file into memory and update it with minimal effort, agnostic to any specifications of the included cards. For this purpose, we use the `Deck` class. Build deckLet's initialize a `Deck` object and add some cards.<jupyter_code>from bulkdata import Deck

deck = Deck()

# add slight variations of the original card we created in the above section
orig_card_str = card.dumps()
for i in range(8):
    # load new card from original card string
    card_var = Card.loads(orig_card_str)
    # just first line
    card_var.resize(8)
    # change name
    card_var.name = "EXAMPL" + str(i)
    # change first field
    card_var[0] += 1
    # change field i
    card_var[i] = "EDITED"
    deck.append(card_var)<jupyter_output><empty_output><jupyter_text>The deck is functionally analogous to a `list` of cards.<jupyter_code>print("Number of cards:", len(deck))
print("First 3 cards:", deck[:3])<jupyter_output>Number of cards: 8
First 3 cards: [Card("EXAMPL0", ['EDITED', 3.14, 'string', '', '', 0, 1, 2]), Card("EXAMPL1", [101, 'EDITED', 'string', '', '', 0, 1, 2]), Card("EXAMPL2", [101, 3.14, 'EDITED', '', '', 0, 1, 2])]<jupyter_text>The `dumps` method returns the deck's string representation, which is the concatenation of its cards' string representations.Just like the `Card` class, printing the `Deck` object uses the `dumps` method, which defaults to _fixed_ format if no format argument is provided.<jupyter_code># these are all analogous

# print(deck.dumps("fixed"))
# print(deck.dumps())
print(deck)<jupyter_output>EXAMPL0 EDITED  3.14    string                  0       1       2
EXAMPL1 101     EDITED  string                  0       1       2
EXAMPL2 101     3.14    EDITED                  0       1       2
EXAMPL3 101     3.14    string  EDITED          0       1       2
EXAMPL4 101     3.14    string          EDITED  0       1       2
EXAMPL5 101     3.14    string                  EDITED  1       2
EXAMPL6 101     3.14    string                  0       EDITED  2
EXAMPL7 101     3.14    string                  0       1       EDITED<jupyter_text>And we can also specify the _free_ format.<jupyter_code>print(deck.dumps("free"))<jupyter_output>EXAMPL0,EDITED,3.14,string, , ,0,1,2
EXAMPL1,101,EDITED,string, , ,0,1,2
EXAMPL2,101,3.14,EDITED, , ,0,1,2
EXAMPL3,101,3.14,string,EDITED, ,0,1,2
EXAMPL4,101,3.14,string, ,EDITED,0,1,2
EXAMPL5,101,3.14,string, , ,EDITED,1,2
EXAMPL6,101,3.14,string, , ,0,EDITED,2
EXAMPL7,101,3.14,string, , ,0,1,EDITED<jupyter_text>Load deckThe driving motivation for this package is to provide the ability to load a BDF file, generated by some external program or process, and update its contents with minimal effort. To do this, we use the `load` classmethod to load the contents of a file object into a `Deck` object.<jupyter_code># the "usage-example.bdf" file is adapted from the pyNastran "testA.bdf" file found here:
# https://github.com/SteveDoyle2/pyNastran/blob/master/pyNastran/bdf/test/unit/testA.bdf
#
# please keep in mind that the original "testA.bdf" was created for testing purposes and
# therefore contains some "rubbish" cards (as does "usage-example.bdf")
bdf_filename = "usage-example.bdf"

with open(bdf_filename) as bdf_file:
    deck = Deck.load(bdf_file)
    
print("Number of cards:", len(deck))<jupyter_output>Number of cards: 143<jupyter_text>Update deckThe BDF file contains an AERO card. Let's find it.<jupyter_code># get all cards with name "AERO"
aero_cards = list(deck.find({"name": "AERO"}))

print("Number of AERO cards:", len(aero_cards))

aero_card = aero_cards[0]

print(aero_card.dumps("free"))<jupyter_output>Number of AERO cards: 1
AERO, , ,1.0,1.0<jupyter_text>The `find` method returns a generator object with all cards from the deck matching the _filter_ argument, in the above case `{"name": "AERO"}`. The _name_ keyword tells the deck to filter for cards matching the specified name. In this case, since we know we there is only once AERO card, we can use the `find_one` method, which returns the first matching card directly. Also, if we are only interested in filtering for the name, we can pass the name string as the _filter_ argument, as a shortcut.<jupyter_code>aero_card = deck.find_one("AERO")<jupyter_output><empty_output><jupyter_text>The AERO card is defined in the __NASTRAN__ manual as follows:```    +------+-------+----------+------+--------+-------+-------+    |   1  |   2   |    3     |   4  |   5    |   6   |   7   |    +======+=======+==========+======+========+=======+=======+    | AERO | ACSID | VELOCITY | REFC | RHOREF | SYMXZ | SYMXY |    +------+-------+----------+------+--------+-------+-------+    | AERO |   3   |   1.3+   | 100. |  1.-5  |   1   |  -1   |    +------+-------+----------+------+--------+-------+-------+``` From this we can see that the AERO card in the deck has blank ACSID and VELOCITY entries; both REFC and RHOREF entries have a value of 1.0; and SYMXZ, SYMXY entries are missing (same as blank).Let's update it to match the example from the manual.<jupyter_code># ACSID
aero_card[0] = 3

# VELOCITY
aero_card[1] = 1.3

# REFC
aero_card[2] = 100.

# RHOREF
aero_card[3] = 1.0e-5

# SYMXZ
aero_card.append(1)

# SYMXY
aero_card.append(-1)

# verify that it was updated in the deck
print(deck.find_one("AERO"))<jupyter_output>AERO    3       1.3     100.    .00001  1       -1<jupyter_text>Alternatively, we can replace the card using the `replace_one` method.<jupyter_code># new AERO card
aero_new = Card("AERO")
aero_new.extend([4, 3.1, 99., 1.0e+5, -1, 1])

deck.replace_one("AERO", aero_new)

# verify that it was updated in the deck
print(deck.find_one("AERO"))<jupyter_output>AERO    4       3.1     99.     100000. -1      1<jupyter_text>In the case of AERO, there is only a single unique card. But what if we want to update several matching cards?The BDF file contains several GRID cards...<jupyter_code>grid_cards = list(deck.find("GRID"))

print("Number of GRID cards:", len(grid_cards))<jupyter_output>Number of GRID cards: 43<jupyter_text>... 43 to be exact. Let's increment each GRID card's first field (NID entry) by 1.<jupyter_code>print("NIDs before update:")
for card in grid_cards:
    print(card[0], end=" ")
    # increment NID
    card[0] += 1
    
print("\n\nNIDs after update:")
for card in deck.find("GRID"):
    print(card[0], end=" ")<jupyter_output>NIDs before update:
1 4 40 41 50 60 120 121 200 1000 1003 1004 1005 1006 1008 1009 1010 1011 1012 2573 2574 2575 2576 16411 16412 16413 16414 16415 16416 16417 16418 16419 10006 10106 10206 10306 10406 10506 10606 10706 10806 12043 31201 

NIDs after update:
2 5 41 42 51 61 121 122 201 1001 1004 1005 1006 1007 1009 1010 1011 1012 1013 2574 2575 2576 2577 16412 16413 16414 16415 16416 16417 16418 16419 16420 10007 10107 10207 10307 10407 10507 10607 10707 10807 12044 31202<jupyter_text>More on filteringUntil this point, we have only filtered for a name, but more complex filtering is also possible.To find cards with specific field values, use the _fields_ keyword:<jupyter_code>filter_ = {
    # name is MAT1
    "name": "MAT1", 
    
    "fields": {
        
        # second field
        "index": 1,
        
        # with value 3.0e7
        "value": 3.0e7
    }
}

for card in deck.find(filter_):
    print(card, end="")<jupyter_output>MAT1    765     3.+7
MAT1    770     3.+7
MAT1    795     3.+7
MAT1    796     3.+7
MAT1    769     3.+7
MAT1    7       3.+7
MAT1    8       3.+7
MAT1    10      3.+7
MAT1    200     3.+7
MAT1    2       3.+7<jupyter_text>To find cards containing a specific value (in any field), use the _contains_ keyword:<jupyter_code>filter_ = {
    # any card containing the "THRU" field
    "contains": "THRU" 
}

for card in deck.find(filter_):
    print(card, end="")<jupyter_output>PLOAD2  13      1.      2100001 THRU    2100003
QBDY3   34      20.             1       THRU    7       BY      2       +0      
+0      10      THRU    40      BY      5       42      45      THRU    +1      
+1      48
QBDY3   500     50000.0         10      THRU    60      BY      10
PLOAD4  510     101     5.                              THRU    112
DDVAL   10      0.1     0.5                                             +0      
+0      1.0     THRU    100.    BY      1.0
ASET1   3       1       THRU    8
ASET1   3       10      THRU    16
SESET   0       1       THRU    10<jupyter_text>Combining the _name_, _fields_, and _contains_ filter keywords:<jupyter_code>filter_ = {
    # name is ASET1
    "name": "ASET1", 
    
    "fields": {
        
        # first field
        "index": 0,
        
        # with value 3
        "value": 3
    },
    
    # contains values 1 and "THRU"
    "contains": [1, "THRU"] 
}

for card in deck.find(filter_):
    print(card, end="")<jupyter_output>ASET1   3       1       THRU    8<jupyter_text>By the way, not providing a filter argument at all, will return all cards in the deck.<jupyter_code>print(len(list(deck.find())))<jupyter_output>143<jupyter_text>Delete cardsThe BDF file contains a "JUNK" card, let's remove it from the deck.<jupyter_code># delete all cards with name "JUNK"
num_deleted = deck.delete("JUNK")

print("Number of cards deleted:", num_deleted)

# verify that all "JUNK" cards have been deleted
no_junk = deck.find_one("JUNK") is None
print(no_junk)<jupyter_output>Number of cards deleted: 1
True<jupyter_text>Just as with the `find` method, we could delete the entire deck if we (recklessly) failed to pass a filter argument. It shouldn't be necessary to demonstrate this...  Dump deckWhen we're ready to write our new and improved deck to file, we use the `dump` method.<jupyter_code>with open("usage-example-updated.bdf", "w") as bdf_file:
    deck.dump(bdf_file)<jupyter_output><empty_output><jupyter_text>Just as with the `dumps` method, the output format of `dump` defaults to _fixed_ but the _free_ format may also be specified.<jupyter_code>with open("usage-example-updated-free.bdf", "w") as bdf_file:
    deck.dump(bdf_file, format="free")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11095897109883124, 'fraction_numerical': 0.06699594112181524, 'mean_word_length': 3.305263157894737, 'pattern_counts': {'":': 12, '<': 127, '<?xml version=': 0, '>': 127, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 12, 'word_list_counts': {'cursed_substrings.json': 25, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '40', 'dir': 'jupyter-structured-clean-dedup', 'id': '654680', 'license': 'MIT', 'n_tokens_mistral': 7602, 'n_tokens_neox': 6663, 'n_words': 2406, 'path': 'examples/bulkdata-usage.ipynb', 'repo_name': 'marcodlk/bulkdata', 'size': '20449'}