mammos_entity.io: reading and writing entities#

[1]:

from pathlib import Path

import mammos_entity as me
import mammos_units as u

Supported file format#

mammos_entity.io can read and write csv and yaml files containing entity like objects (entities, quantities, or other array-like data). For entities and quantities information about ontology and units are included as additional metadata. Details of the file formats are explained in the `mammos-entity.io api reference <https://mammos-project.github.io/mammos/api/mammos_entity.io.html>`__.

We create some artificial data that we can write to file:

[2]:

Ms = me.Ms([600, 650, 700], "kA/m")
T = me.T([1, 2, 3])
theta_angle = [0, 0.5, 0.7] * u.rad
demag_factor = me.Entity("DemagnetizingFactor", [1 / 3, 1 / 3, 1 / 3])
comments = ["Some comment", "Some other comment", "A third comment"]

MaMMoS CSV#

Writing#

We can write data to a csv file as shown in the following cell. Names of the keyword arguments determine column names in the file.

[3]:

me.io.entities_to_file(
    "example.csv",
    "Example csv file.\nThe description can have multiple lines.\n\nLines can also be empty.",
    Ms=Ms,
    T=T,
    angle=theta_angle,
    demag_factor=demag_factor,
    comment=comments,
)

This has produced the following file:

[4]:

print(Path("example.csv").read_text())

#mammos csv v2
#----------------------------------------
# Example csv file.
# The description can have multiple lines.
#
# Lines can also be empty.
#----------------------------------------
#SpontaneousMagnetization,ThermodynamicTemperature,,DemagnetizingFactor,
#https://w3id.org/emmo/domain/magnetic_material#EMMO_032731f8-874d-5efb-9c9d-6dafaa17ef25,https://w3id.org/emmo#EMMO_affe07e4_e9bc_4852_86c6_69e26182a17f,,https://w3id.org/emmo/domain/magnetic_material#EMMO_0f2b5cc9-d00a-5030-8448-99ba6b7dfd1e,
#kA / m,K,rad,,
Ms,T,angle,demag_factor,comment
600.0,1.0,0.0,0.3333333333333333,Some comment
650.0,2.0,0.5,0.3333333333333333,Some other comment
700.0,3.0,0.7,0.3333333333333333,A third comment

Reading#

We can read it back in and get a container object (called EntityCollection) containing all columns:

[5]:

content = me.io.entities_from_file("example.csv")
content

[5]:

EntityCollection(
    Ms=Entity(ontology_label='SpontaneousMagnetization', value=array([600., 650., 700.]), unit='kA / m'),
    T=Entity(ontology_label='ThermodynamicTemperature', value=array([1., 2., 3.]), unit='K'),
    angle=<Quantity [0. , 0.5, 0.7] rad>,
    demag_factor=Entity(ontology_label='DemagnetizingFactor', value=array([0.33333333, 0.33333333, 0.33333333])),
    comment=array(['Some comment', 'Some other comment', 'A third comment'],
      dtype=object),
)

The recommended way of accessing the data is by using the individual elements. This preserves the correct data type:

[6]:

content.Ms

[6]:
SpontaneousMagnetization(value=[600. 650. 700.], unit=kA / m)
[7]:

content.T

[7]:
ThermodynamicTemperature(value=[1. 2. 3.], unit=K)
[8]:

content.angle

[8]:
$[0,~0.5,~0.7] \; \mathrm{rad}$
[9]:

content.demag_factor

[9]:
DemagnetizingFactor(value=[0.33333333 0.33333333 0.33333333])
[10]:

content.comment

[10]:

array(['Some comment', 'Some other comment', 'A third comment'],
      dtype=object)

We can also get a pandas dataframe of the data we have read. This is designed as a convenience functions but due to limitation of pandas we loose ontology information. This is why we recommend using the individual elements directly where possible. The columns names consist of short name and units (where columns have a unit):

[11]:

content.to_dataframe()

[11]:
Ms (kA / m) T (K) angle (rad) demag_factor comment
0 600.0 1.0 0.0 0.333333 Some comment
1 650.0 2.0 0.5 0.333333 Some other comment
2 700.0 3.0 0.7 0.333333 A third comment

We can also get a dataframe without units in the column names:

[12]:

content.to_dataframe(include_units=False)

[12]:
Ms T angle demag_factor comment
0 600.0 1.0 0.0 0.333333 Some comment
1 650.0 2.0 0.5 0.333333 Some other comment
2 700.0 3.0 0.7 0.333333 A third comment

Reading with pandas#

If we only need the numerical data but not the entity information, we can also read the csv file with pandas:

[13]:

import pandas as pd

pd.read_csv("example.csv", comment="#")

[13]:
Ms T angle demag_factor comment
0 600.0 1.0 0.0 0.333333 Some comment
1 650.0 2.0 0.5 0.333333 Some other comment
2 700.0 3.0 0.7 0.333333 A third comment

MaMMoS YAML#

Writing#

We can write data to a yaml file (extension .yaml or .yml) as shown in the following cell. Names of the keyword arguments determine keys under data: in the file.

[14]:

me.io.entities_to_file(
    "example.yaml",
    "Example yaml file.\nThe description can have multiple lines.\n\nLines can also be empty.",
    Ms=Ms,
    T=T,
    angle=theta_angle,
    demag_factor=demag_factor,
    comment=comments,
)

This has produced the following file:

[15]:

print(Path("example.yaml").read_text())

metadata:
  version: v1
  description: |-
    Example yaml file.
    The description can have multiple lines.

    Lines can also be empty.
data:
  Ms:
    ontology_label: SpontaneousMagnetization
    ontology_iri: https://w3id.org/emmo/domain/magnetic_material#EMMO_032731f8-874d-5efb-9c9d-6dafaa17ef25
    unit: kA / m
    value: [600.0, 650.0, 700.0]
  T:
    ontology_label: ThermodynamicTemperature
    ontology_iri: https://w3id.org/emmo#EMMO_affe07e4_e9bc_4852_86c6_69e26182a17f
    unit: K
    value: [1.0, 2.0, 3.0]
  angle:
    ontology_label: null
    ontology_iri: null
    unit: rad
    value: [0.0, 0.5, 0.7]
  demag_factor:
    ontology_label: DemagnetizingFactor
    ontology_iri: https://w3id.org/emmo/domain/magnetic_material#EMMO_0f2b5cc9-d00a-5030-8448-99ba6b7dfd1e
    unit: ''
    value: [0.3333333333333333, 0.3333333333333333, 0.3333333333333333]
  comment:
    ontology_label: null
    ontology_iri: null
    unit: null
    value: [Some comment, Some other comment, A third comment]

Reading#

We can read it back in and get a container object (called EntityCollection) containing all columns:

[16]:

content = me.io.entities_from_file("example.yaml")
content

[16]:

EntityCollection(
    Ms=Entity(ontology_label='SpontaneousMagnetization', value=array([600., 650., 700.]), unit='kA / m'),
    T=Entity(ontology_label='ThermodynamicTemperature', value=array([1., 2., 3.]), unit='K'),
    angle=<Quantity [0. , 0.5, 0.7] rad>,
    demag_factor=Entity(ontology_label='DemagnetizingFactor', value=array([0.33333333, 0.33333333, 0.33333333])),
    comment=['Some comment', 'Some other comment', 'A third comment'],
)

We can access elements as show for the csv file before, e.g.

[17]:

content.to_dataframe()

[17]:
Ms (kA / m) T (K) angle (rad) demag_factor comment
0 600.0 1.0 0.0 0.333333 Some comment
1 650.0 2.0 0.5 0.333333 Some other comment
2 700.0 3.0 0.7 0.333333 A third comment

Entities of different shape and length#

Unlike csv the yaml format supports entities with different lengths and shape.

[18]:

me.io.entities_to_file("example2.yml", T=me.T([10, 20, 30]), Tc=me.Tc(300), Ms=me.Ms([[1, 2], [3, 4]]))

[19]:

print(Path("example2.yml").read_text())

metadata:
  version: v1
  description: null
data:
  T:
    ontology_label: ThermodynamicTemperature
    ontology_iri: https://w3id.org/emmo#EMMO_affe07e4_e9bc_4852_86c6_69e26182a17f
    unit: K
    value: [10.0, 20.0, 30.0]
  Tc:
    ontology_label: CurieTemperature
    ontology_iri: https://w3id.org/emmo#EMMO_6b5af5a8_a2d8_4353_a1d6_54c9f778343d
    unit: K
    value: 300.0
  Ms:
    ontology_label: SpontaneousMagnetization
    ontology_iri: https://w3id.org/emmo/domain/magnetic_material#EMMO_032731f8-874d-5efb-9c9d-6dafaa17ef25
    unit: A / m
    value: [[1.0, 2.0], [3.0, 4.0]]


[20]:

content = me.io.entities_from_file("example2.yml")
content

[20]:

EntityCollection(
    T=Entity(ontology_label='ThermodynamicTemperature', value=array([10., 20., 30.]), unit='K'),
    Tc=Entity(ontology_label='CurieTemperature', value=np.float64(300.0), unit='K'),
    Ms=Entity(ontology_label='SpontaneousMagnetization', value=array([[1., 2.],
       [3., 4.]]), unit='A / m'),
)

We can acces individual elements of the returned object as before.

[21]:

content.T

[21]:
ThermodynamicTemperature(value=[10. 20. 30.], unit=K)
[22]:

content.Tc

[22]:
CurieTemperature(value=300.0, unit=K)
[23]:

content.Ms

[23]:
SpontaneousMagnetization(value=[[1. 2.]
 [3. 4.]], unit=A / m)

Conversion to a dataframe is however generally not possible as the shapes are incompatible with a pandas.DataFrame.

Converting unformatted files and tables to mammos csv#

Users may wish to update other files to the mammos csv format in order to make use of the additional functionality. Details of the file format are explained in the `mammos-entity.io api reference <https://mammos-project.github.io/mammos/api/mammos_entity.io.html>`__.

Converting your “raw” data into this format involves three main steps:

  1. Load your file into python (e.g. with pandas).

  2. Create an Entity, quanity, or similar out of each column (by assigning the correct ontology term and/or units).

  3. Export the result with entities_to_csv.

First let’s create a file so we can see an example of how to do the conversion. We will create the following structure:

1 10.0 1.6083568305976572 -16778187.088808443
1 9.0 1.6083393931987826 -15498304.121589921
...

This file is quite basic, in particular, there no headers, no units, no ontology information, and the use of the space as separator rather than a comma.

Only the user knows what each of the columns are. In this example first column is the configuration type, the second column is the value of \(\mu_0 H_{\mathsf{ext}}\) in Tesla, the third column is the magnetic polarisation in Tesla and the last column is the energy density in J/m\(^3\).

[24]:

Path("example.dat").write_text("""\
1 10.0 1.6083568305976572 -16778187.088808443
1 9.0 1.6083393931987826 -15498304.121589921
1 8.0 1.6083184361075116 -14218436.37373519
1 7.0 1.608292941666901 -12938587.029585946
1 6.0 1.6082614950059932 -11658760.230932372
""")

[24]:

224

We can use pandas to read the file into python:

[25]:

df = pd.read_csv("example.dat", sep=" ", names=["configuration_type", "mu0_Hext", "Js", "energy_density"])
df

[25]:
configuration_type mu0_Hext Js energy_density
0 1 10.0 1.608357 -1.677819e+07
1 1 9.0 1.608339 -1.549830e+07
2 1 8.0 1.608318 -1.421844e+07
3 1 7.0 1.608293 -1.293859e+07
4 1 6.0 1.608261 -1.165876e+07

To rewrite this in the mammos csv format, we then need to associate each column with an entity, quantity, or another python object. Now is also time to do any data manipulation (such as changing units).

In this example we:

  • Convert configuration type to a numpy array.

  • Convert magnetic flux density (\(\mu_0 H_{\mathsf{ext}}\)) to the external magnetic field Entity using mammos_units for the relevant conversions.

  • Convert magnetic polarisation to the corresponding entity.

  • Convert energy density to the corresponding entity.

[26]:

configuration_type = df["configuration_type"].to_numpy()
H = me.Entity(
    ontology_label="ExternalMagneticField",
    value=(df["mu0_Hext"].to_numpy() * u.T).to(u.A / u.m, equivalencies=u.magnetic_flux_field()),
    unit=u.A / u.m,
)
Js = me.Entity(
    ontology_label="MagneticPolarisation",
    value=df["Js"],
    unit=u.T,
)
energy_density = me.Entity(ontology_label="EnergyDensity", value=df["energy_density"], unit=u.J / u.m**3)

We can now write the mammos csv:

[27]:

me.io.entities_to_file(
    "example.csv",
    "This file contains...",
    configuration_type=configuration_type,
    H=H,
    Js=Js,
    energy_density=energy_density,
)

Looking at the file produced we can see the data is now in the correct format with the ontology information included:

[28]:

print(Path("example.csv").read_text())

#mammos csv v2
#----------------------------------------
# This file contains...
#----------------------------------------
#,ExternalMagneticField,MagneticPolarisation,EnergyDensity
#,https://w3id.org/emmo/domain/magnetic_material#EMMO_da08f0d3-fe19-58bc-8fb6-ecc8992d5eb3,https://w3id.org/emmo#EMMO_74a096dd_cc83_4c7e_b704_0541620ff18d,https://w3id.org/emmo/domain/magnetic_material#EMMO_56258d3a-f2ee-554e-af99-499dd8620457
#,A / m,T,J / m3
configuration_type,H,Js,energy_density
1,7957747.150262763,1.6083568305976572,-16778187.088808443
1,7161972.435236487,1.6083393931987826,-15498304.12158992
1,6366197.72021021,1.6083184361075116,-14218436.37373519
1,5570423.005183934,1.608292941666901,-12938587.029585946
1,4774648.290157658,1.6082614950059932,-11658760.230932372